[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:15:12.513023 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:45553
I20260709 12:15:12.514600 17157 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:15:12.515702 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:12.533815 17166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:12.533893 17164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:12.534008 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:13.740118 17165 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:15:13.740365 17157 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:15:13.744890 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:13.745115 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:13.745375 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599313745341 us; error 0 us; skew 500 ppm
I20260709 12:15:13.746212 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:13.752650 17157 webserver.cc:533] Webserver started at http://127.16.193.126:40587/ using document root <none> and password file <none>
I20260709 12:15:13.753515 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:13.753746 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:13.754200 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:13.759969 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "1561d7861311457ba94c83ec0f7ab08d"
format_stamp: "Formatted at 2026-07-09 12:15:13 on dist-test-slave-n640"
I20260709 12:15:13.767671 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:15:13.772771 17172 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:13.773849 17157 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:13.774206 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "1561d7861311457ba94c83ec0f7ab08d"
format_stamp: "Formatted at 2026-07-09 12:15:13 on dist-test-slave-n640"
I20260709 12:15:13.774545 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:13.834245 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:13.835880 17157 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:15:13.836351 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:13.907637 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:45553
I20260709 12:15:13.907711 17223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:45553 every 8 connection(s)
I20260709 12:15:13.915421 17224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:13.936934 17224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: Bootstrap starting.
I20260709 12:15:13.942744 17224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:13.944589 17224 log.cc:826] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: Log is configured to *not* fsync() on all Append() calls
I20260709 12:15:13.949086 17224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: No bootstrap required, opened a new log
I20260709 12:15:13.969614 17224 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER }
I20260709 12:15:13.970172 17224 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:13.970383 17224 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1561d7861311457ba94c83ec0f7ab08d, State: Initialized, Role: FOLLOWER
I20260709 12:15:13.971273 17224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER }
I20260709 12:15:13.971796 17224 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:13.972114 17224 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:13.972401 17224 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:13.976421 17224 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER }
I20260709 12:15:13.977234 17224 leader_election.cc:304] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1561d7861311457ba94c83ec0f7ab08d; no voters: 
I20260709 12:15:13.978890 17224 leader_election.cc:290] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:13.979223 17227 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:13.981942 17227 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 1 LEADER]: Becoming Leader. State: Replica: 1561d7861311457ba94c83ec0f7ab08d, State: Running, Role: LEADER
I20260709 12:15:13.982831 17227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER }
I20260709 12:15:13.983928 17224 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:13.994767 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1561d7861311457ba94c83ec0f7ab08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER } }
I20260709 12:15:13.994822 17229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1561d7861311457ba94c83ec0f7ab08d. Latest consensus state: current_term: 1 leader_uuid: "1561d7861311457ba94c83ec0f7ab08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1561d7861311457ba94c83ec0f7ab08d" member_type: VOTER } }
I20260709 12:15:13.995656 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:13.995716 17229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:13.998805 17237 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:14.009651 17237 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:14.011987 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:14.025219 17237 catalog_manager.cc:1383] Generated new cluster ID: 9b641420da6a46df868571ecb38fc233
I20260709 12:15:14.025517 17237 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:14.082651 17237 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:14.084868 17237 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:14.110453 17237 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: Generated new TSK 0
I20260709 12:15:14.111424 17237 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:14.142752 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:14.151053 17246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:14.152253 17247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:14.156085 17249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:14.156162 17157 server_base.cc:1061] running on GCE node
I20260709 12:15:14.157089 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:14.157275 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:14.157483 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599314157444 us; error 0 us; skew 500 ppm
I20260709 12:15:14.158077 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:14.162379 17157 webserver.cc:533] Webserver started at http://127.16.193.65:42237/ using document root <none> and password file <none>
I20260709 12:15:14.162918 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:14.163120 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:14.163434 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:14.164532 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "35f65a8690b24bc3a884546c776a3360"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.170166 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:15:14.174821 17254 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.175668 17157 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:15:14.175956 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "35f65a8690b24bc3a884546c776a3360"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.176245 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:14.200145 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:14.201439 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:14.204170 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:14.208674 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:14.208917 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.209232 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:14.209393 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.315480 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:36889
I20260709 12:15:14.315569 17316 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:36889 every 8 connection(s)
I20260709 12:15:14.323336 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:14.338284 17321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:14.341822 17322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:14.350780 17157 server_base.cc:1061] running on GCE node
W20260709 12:15:14.351715 17324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:14.352892 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:14.353082 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:14.353281 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599314353265 us; error 0 us; skew 500 ppm
I20260709 12:15:14.353801 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:14.358157 17157 webserver.cc:533] Webserver started at http://127.16.193.66:35085/ using document root <none> and password file <none>
I20260709 12:15:14.358861 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:14.359043 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:14.359210 17317 heartbeater.cc:344] Connected to a master server at 127.16.193.126:45553
I20260709 12:15:14.359372 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:14.359701 17317 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:14.360914 17317 heartbeater.cc:507] Master 127.16.193.126:45553 requested a full tablet report, sending...
I20260709 12:15:14.360862 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.364568 17189 ts_manager.cc:194] Registered new tserver with Master: 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889)
I20260709 12:15:14.367467 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:15:14.367575 17189 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46804
I20260709 12:15:14.372283 17329 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.373528 17157 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:15:14.373823 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.374130 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:14.393416 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:14.394822 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:14.396982 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:14.400027 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:14.400252 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.400494 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:14.400666 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.493371 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:33015
I20260709 12:15:14.493729 17391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:33015 every 8 connection(s)
I20260709 12:15:14.500336 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:14.508689 17395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:14.510516 17396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:14.519001 17392 heartbeater.cc:344] Connected to a master server at 127.16.193.126:45553
I20260709 12:15:14.519505 17392 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:14.521459 17392 heartbeater.cc:507] Master 127.16.193.126:45553 requested a full tablet report, sending...
W20260709 12:15:14.523042 17398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:14.523928 17189 ts_manager.cc:194] Registered new tserver with Master: f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:14.524700 17157 server_base.cc:1061] running on GCE node
I20260709 12:15:14.525627 17189 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46816
I20260709 12:15:14.525794 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:14.526086 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:14.526289 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599314526272 us; error 0 us; skew 500 ppm
I20260709 12:15:14.527016 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:14.529858 17157 webserver.cc:533] Webserver started at http://127.16.193.67:43791/ using document root <none> and password file <none>
I20260709 12:15:14.530457 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:14.530694 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:14.530995 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:14.532362 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "eecf17b6d17242b3bef574c2790ff838"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.537962 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:15:14.542150 17403 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.543017 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:15:14.543346 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "eecf17b6d17242b3bef574c2790ff838"
format_stamp: "Formatted at 2026-07-09 12:15:14 on dist-test-slave-n640"
I20260709 12:15:14.543675 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:14.560072 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:14.561637 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:14.564117 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:14.567616 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:14.567874 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.568146 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:14.568359 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:14.654837 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:41171
I20260709 12:15:14.654887 17465 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:41171 every 8 connection(s)
I20260709 12:15:14.672178 17466 heartbeater.cc:344] Connected to a master server at 127.16.193.126:45553
I20260709 12:15:14.672545 17466 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:14.673454 17466 heartbeater.cc:507] Master 127.16.193.126:45553 requested a full tablet report, sending...
I20260709 12:15:14.675314 17189 ts_manager.cc:194] Registered new tserver with Master: eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:14.676193 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015866137s
I20260709 12:15:14.676743 17189 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46826
I20260709 12:15:14.727913 17189 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46830:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:14.731608 17189 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:14.841797 17357 tablet_service.cc:1511] Processing CreateTablet for tablet 184227df1321443489980c96dbba4e5f (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:14.841969 17282 tablet_service.cc:1511] Processing CreateTablet for tablet 184227df1321443489980c96dbba4e5f (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:14.844095 17356 tablet_service.cc:1511] Processing CreateTablet for tablet 70c16bb43a874fc19023a6aa38512b59 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:14.845738 17354 tablet_service.cc:1511] Processing CreateTablet for tablet 86532393c7d645a587c7e9c076b75f66 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:14.846057 17355 tablet_service.cc:1511] Processing CreateTablet for tablet c05ef912897f4cb2ac5a9c41ef27bc37 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:14.847494 17350 tablet_service.cc:1511] Processing CreateTablet for tablet d0494158f31647c99836512eef3c5da8 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:14.843721 17282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 184227df1321443489980c96dbba4e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.847865 17353 tablet_service.cc:1511] Processing CreateTablet for tablet 8de41642be634fb2af89f5788d4847aa (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:14.848011 17352 tablet_service.cc:1511] Processing CreateTablet for tablet d7b6545b38e84924b860ddb9d791bcb5 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:14.849521 17351 tablet_service.cc:1511] Processing CreateTablet for tablet bbb00b52652b49589deaffba5311f162 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:14.850423 17281 tablet_service.cc:1511] Processing CreateTablet for tablet 70c16bb43a874fc19023a6aa38512b59 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:14.851856 17281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c16bb43a874fc19023a6aa38512b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.851557 17349 tablet_service.cc:1511] Processing CreateTablet for tablet c36a4b0893c5471ca03b615379c68d82 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:14.852972 17357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 184227df1321443489980c96dbba4e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.858997 17352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b6545b38e84924b860ddb9d791bcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.860953 17280 tablet_service.cc:1511] Processing CreateTablet for tablet c05ef912897f4cb2ac5a9c41ef27bc37 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:14.862202 17280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05ef912897f4cb2ac5a9c41ef27bc37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.863883 17349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c36a4b0893c5471ca03b615379c68d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.867859 17356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c16bb43a874fc19023a6aa38512b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.873703 17279 tablet_service.cc:1511] Processing CreateTablet for tablet 86532393c7d645a587c7e9c076b75f66 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:14.875332 17279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86532393c7d645a587c7e9c076b75f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.875797 17351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb00b52652b49589deaffba5311f162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.880283 17354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86532393c7d645a587c7e9c076b75f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.886940 17278 tablet_service.cc:1511] Processing CreateTablet for tablet 8de41642be634fb2af89f5788d4847aa (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:14.886971 17355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05ef912897f4cb2ac5a9c41ef27bc37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.889324 17277 tablet_service.cc:1511] Processing CreateTablet for tablet d7b6545b38e84924b860ddb9d791bcb5 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:14.891065 17275 tablet_service.cc:1511] Processing CreateTablet for tablet c36a4b0893c5471ca03b615379c68d82 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:14.898865 17282 tablet_service.cc:1511] Processing CreateTablet for tablet bbb00b52652b49589deaffba5311f162 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:14.903064 17275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c36a4b0893c5471ca03b615379c68d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.905158 17278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de41642be634fb2af89f5788d4847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.904608 17483 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:14.891290 17276 tablet_service.cc:1511] Processing CreateTablet for tablet d0494158f31647c99836512eef3c5da8 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:14.907888 17350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0494158f31647c99836512eef3c5da8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.915652 17353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de41642be634fb2af89f5788d4847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.927338 17276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0494158f31647c99836512eef3c5da8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.948483 17277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b6545b38e84924b860ddb9d791bcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.956178 17282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb00b52652b49589deaffba5311f162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:14.980556 17483 tablet_bootstrap.cc:654] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:14.984942 17430 tablet_service.cc:1511] Processing CreateTablet for tablet 70c16bb43a874fc19023a6aa38512b59 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:14.992255 17429 tablet_service.cc:1511] Processing CreateTablet for tablet c05ef912897f4cb2ac5a9c41ef27bc37 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:14.991168 17431 tablet_service.cc:1511] Processing CreateTablet for tablet 184227df1321443489980c96dbba4e5f (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:14.993008 17428 tablet_service.cc:1511] Processing CreateTablet for tablet 86532393c7d645a587c7e9c076b75f66 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:14.998994 17427 tablet_service.cc:1511] Processing CreateTablet for tablet 8de41642be634fb2af89f5788d4847aa (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:14.998509 17484 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
W20260709 12:15:15.005133 17236 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:15.006069 17426 tablet_service.cc:1511] Processing CreateTablet for tablet d7b6545b38e84924b860ddb9d791bcb5 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:15.006744 17430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c16bb43a874fc19023a6aa38512b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.015219 17429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05ef912897f4cb2ac5a9c41ef27bc37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.019708 17425 tablet_service.cc:1511] Processing CreateTablet for tablet bbb00b52652b49589deaffba5311f162 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:15.023353 17424 tablet_service.cc:1511] Processing CreateTablet for tablet d0494158f31647c99836512eef3c5da8 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:15.019718 17423 tablet_service.cc:1511] Processing CreateTablet for tablet c36a4b0893c5471ca03b615379c68d82 (DEFAULT_TABLE table=test-workload [id=7a19856fd96e437e8ad12fef3f267f1f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:15.029343 17427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de41642be634fb2af89f5788d4847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.030256 17425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb00b52652b49589deaffba5311f162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.035420 17424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0494158f31647c99836512eef3c5da8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.036762 17428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86532393c7d645a587c7e9c076b75f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.042817 17426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b6545b38e84924b860ddb9d791bcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.044086 17423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c36a4b0893c5471ca03b615379c68d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.045926 17431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 184227df1321443489980c96dbba4e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:15.073078 17484 tablet_bootstrap.cc:654] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.080291 17483 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.080870 17483 ts_tablet_manager.cc:1403] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.191s	user 0.012s	sys 0.007s
I20260709 12:15:15.102090 17487 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.102371 17483 raft_consensus.cc:359] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.107947 17484 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.106298 17483 raft_consensus.cc:385] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.109952 17483 raft_consensus.cc:740] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.109807 17484 ts_tablet_manager.cc:1403] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.128s	user 0.021s	sys 0.007s
I20260709 12:15:15.124028 17483 consensus_queue.cc:260] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.127185 17484 raft_consensus.cc:359] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.130354 17484 raft_consensus.cc:385] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.131783 17484 raft_consensus.cc:740] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.132884 17484 consensus_queue.cc:260] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.136917 17317 heartbeater.cc:499] Master 127.16.193.126:45553 was elected leader, sending a full tablet report...
I20260709 12:15:15.137300 17487 tablet_bootstrap.cc:654] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.135314 17483 ts_tablet_manager.cc:1434] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.054s	user 0.011s	sys 0.005s
I20260709 12:15:15.139119 17483 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.145553 17392 heartbeater.cc:499] Master 127.16.193.126:45553 was elected leader, sending a full tablet report...
I20260709 12:15:15.149156 17484 ts_tablet_manager.cc:1434] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 12:15:15.150166 17484 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.155375 17489 raft_consensus.cc:493] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.156412 17489 raft_consensus.cc:515] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.162814 17489 leader_election.cc:290] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.172216 17484 tablet_bootstrap.cc:654] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.177063 17487 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.177557 17487 ts_tablet_manager.cc:1403] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.008s
I20260709 12:15:15.178416 17483 tablet_bootstrap.cc:654] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.188901 17487 raft_consensus.cc:359] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.198582 17487 raft_consensus.cc:385] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.201642 17487 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.203354 17487 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.204949 17484 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.205432 17484 ts_tablet_manager.cc:1403] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.005s
I20260709 12:15:15.205791 17483 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.206255 17483 ts_tablet_manager.cc:1403] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.005s
I20260709 12:15:15.208191 17484 raft_consensus.cc:359] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.208902 17484 raft_consensus.cc:385] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.208932 17483 raft_consensus.cc:359] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.209293 17484 raft_consensus.cc:740] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.209666 17483 raft_consensus.cc:385] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.209997 17483 raft_consensus.cc:740] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.210202 17484 consensus_queue.cc:260] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.212803 17484 ts_tablet_manager.cc:1434] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:15:15.213680 17484 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.210772 17483 consensus_queue.cc:260] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.216485 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.217238 17483 ts_tablet_manager.cc:1434] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:15:15.218201 17483 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
W20260709 12:15:15.219864 17331 leader_election.cc:343] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.220431 17484 tablet_bootstrap.cc:654] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.224152 17466 heartbeater.cc:499] Master 127.16.193.126:45553 was elected leader, sending a full tablet report...
I20260709 12:15:15.226738 17484 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.226570 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.227196 17484 ts_tablet_manager.cc:1403] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.005s
W20260709 12:15:15.228106 17331 leader_election.cc:343] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.228536 17331 leader_election.cc:304] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:15.229218 17487 ts_tablet_manager.cc:1434] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.001s
I20260709 12:15:15.229322 17489 raft_consensus.cc:2749] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:15.230201 17487 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.231040 17484 raft_consensus.cc:359] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.231781 17484 raft_consensus.cc:385] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.232067 17484 raft_consensus.cc:740] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.232826 17484 consensus_queue.cc:260] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.235297 17484 ts_tablet_manager.cc:1434] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:15.236218 17484 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.237479 17487 tablet_bootstrap.cc:654] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.242421 17483 tablet_bootstrap.cc:654] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.242551 17484 tablet_bootstrap.cc:654] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:15.250770 17393 tablet.cc:2406] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:15.251991 17487 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.252430 17487 ts_tablet_manager.cc:1403] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.004s
I20260709 12:15:15.255921 17487 raft_consensus.cc:359] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.256608 17487 raft_consensus.cc:385] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.256915 17487 raft_consensus.cc:740] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.257359 17484 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.258316 17484 ts_tablet_manager.cc:1403] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:15:15.259858 17483 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.260277 17483 ts_tablet_manager.cc:1403] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.004s
I20260709 12:15:15.261101 17495 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.261607 17495 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.263203 17483 raft_consensus.cc:359] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.263998 17483 raft_consensus.cc:385] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.264259 17483 raft_consensus.cc:740] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.264925 17483 consensus_queue.cc:260] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.265431 17495 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:15.267529 17483 ts_tablet_manager.cc:1434] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 12:15:15.268435 17483 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.257701 17487 consensus_queue.cc:260] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.277956 17487 ts_tablet_manager.cc:1434] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:15:15.278857 17487 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.285934 17487 tablet_bootstrap.cc:654] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.276068 17484 raft_consensus.cc:359] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.290155 17484 raft_consensus.cc:385] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.290477 17484 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.291473 17483 tablet_bootstrap.cc:654] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.291174 17484 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.295778 17484 ts_tablet_manager.cc:1434] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 12:15:15.296761 17484 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.309531 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
W20260709 12:15:15.311259 17405 leader_election.cc:343] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.316972 17487 tablet_bootstrap.cc:492] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.317440 17487 ts_tablet_manager.cc:1403] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.008s
I20260709 12:15:15.318567 17484 tablet_bootstrap.cc:654] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.321448 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.321162 17487 raft_consensus.cc:359] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.321938 17483 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.322099 17487 raft_consensus.cc:385] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.322386 17483 ts_tablet_manager.cc:1403] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.054s	user 0.007s	sys 0.016s
I20260709 12:15:15.322540 17487 raft_consensus.cc:740] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
W20260709 12:15:15.323041 17405 leader_election.cc:343] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.323485 17405 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838; no voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:15.324256 17495 raft_consensus.cc:2749] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:15.323560 17487 consensus_queue.cc:260] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.325480 17483 raft_consensus.cc:359] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.326155 17483 raft_consensus.cc:385] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.328833 17487 ts_tablet_manager.cc:1434] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.005s
I20260709 12:15:15.333869 17484 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.334052 17487 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.334324 17484 ts_tablet_manager.cc:1403] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.006s
I20260709 12:15:15.337262 17484 raft_consensus.cc:359] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.338034 17484 raft_consensus.cc:385] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.338339 17484 raft_consensus.cc:740] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.329032 17483 raft_consensus.cc:740] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.339186 17484 consensus_queue.cc:260] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.340123 17483 consensus_queue.cc:260] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.340771 17487 tablet_bootstrap.cc:654] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.342437 17483 ts_tablet_manager.cc:1434] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:15:15.343475 17483 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.349766 17483 tablet_bootstrap.cc:654] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.352124 17484 ts_tablet_manager.cc:1434] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:15:15.359959 17484 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.368517 17484 tablet_bootstrap.cc:654] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.373976 17487 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.374538 17487 ts_tablet_manager.cc:1403] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.017s
I20260709 12:15:15.374964 17483 tablet_bootstrap.cc:492] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.375442 17483 ts_tablet_manager.cc:1403] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.016s
I20260709 12:15:15.378332 17483 raft_consensus.cc:359] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.378507 17487 raft_consensus.cc:359] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.379081 17483 raft_consensus.cc:385] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.379531 17487 raft_consensus.cc:385] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.379567 17483 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.380097 17487 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.380587 17483 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.383276 17483 ts_tablet_manager.cc:1434] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:15.381173 17487 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.385746 17487 ts_tablet_manager.cc:1434] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:15:15.386767 17487 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.390028 17484 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.390573 17484 ts_tablet_manager.cc:1403] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.004s
I20260709 12:15:15.393291 17487 tablet_bootstrap.cc:654] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.393869 17484 raft_consensus.cc:359] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.394646 17484 raft_consensus.cc:385] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.394966 17488 raft_consensus.cc:493] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.394968 17484 raft_consensus.cc:740] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.395643 17488 raft_consensus.cc:515] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.396063 17484 consensus_queue.cc:260] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.398216 17488 leader_election.cc:290] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.398860 17484 ts_tablet_manager.cc:1434] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:15:15.404601 17484 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.399525 17483 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.415791 17483 tablet_bootstrap.cc:654] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.419183 17484 tablet_bootstrap.cc:654] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.426198 17483 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.426208 17487 tablet_bootstrap.cc:492] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.426733 17483 ts_tablet_manager.cc:1403] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.043s	user 0.006s	sys 0.004s
I20260709 12:15:15.426846 17487 ts_tablet_manager.cc:1403] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.008s
I20260709 12:15:15.429455 17483 raft_consensus.cc:359] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.429811 17487 raft_consensus.cc:359] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.430550 17487 raft_consensus.cc:385] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.430868 17487 raft_consensus.cc:740] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.431994 17484 tablet_bootstrap.cc:492] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.431638 17487 consensus_queue.cc:260] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.432471 17484 ts_tablet_manager.cc:1403] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.004s
I20260709 12:15:15.433856 17487 ts_tablet_manager.cc:1434] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:15:15.434829 17487 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.436995 17484 raft_consensus.cc:359] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.437690 17484 raft_consensus.cc:385] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.437981 17484 raft_consensus.cc:740] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.440647 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.430178 17483 raft_consensus.cc:385] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.441277 17487 tablet_bootstrap.cc:654] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.441327 17367 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.441510 17483 raft_consensus.cc:740] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.442344 17483 consensus_queue.cc:260] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.443032 17256 leader_election.cc:304] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.438687 17484 consensus_queue.cc:260] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.446115 17484 ts_tablet_manager.cc:1434] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:15:15.447005 17484 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.453459 17484 tablet_bootstrap.cc:654] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.455683 17488 raft_consensus.cc:2804] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.456094 17488 raft_consensus.cc:493] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.456449 17488 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.460904 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
W20260709 12:15:15.462354 17256 leader_election.cc:343] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.463150 17488 raft_consensus.cc:515] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.465448 17488 leader_election.cc:290] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.466631 17487 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.467106 17487 ts_tablet_manager.cc:1403] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.033s	user 0.000s	sys 0.020s
I20260709 12:15:15.467470 17483 ts_tablet_manager.cc:1434] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.003s
I20260709 12:15:15.467548 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.468426 17483 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.467039 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.469189 17367 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.474363 17483 tablet_bootstrap.cc:654] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:15.475579 17256 leader_election.cc:343] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.476998 17367 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.476756 17487 raft_consensus.cc:359] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.477557 17487 raft_consensus.cc:385] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.477871 17487 raft_consensus.cc:740] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.478492 17256 leader_election.cc:304] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:15.478794 17487 consensus_queue.cc:260] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.479421 17488 raft_consensus.cc:2804] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.479902 17488 raft_consensus.cc:697] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:15.480656 17488 consensus_queue.cc:237] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.481338 17487 ts_tablet_manager.cc:1434] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:15:15.482728 17487 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.491277 17483 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.491751 17483 ts_tablet_manager.cc:1403] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 12:15:15.494436 17483 raft_consensus.cc:359] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.496359 17487 tablet_bootstrap.cc:654] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.498706 17484 tablet_bootstrap.cc:492] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.499111 17484 ts_tablet_manager.cc:1403] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.000s
I20260709 12:15:15.501624 17484 raft_consensus.cc:359] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.502074 17487 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.502250 17484 raft_consensus.cc:385] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.502550 17484 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.502565 17487 ts_tablet_manager.cc:1403] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.007s
I20260709 12:15:15.505522 17487 raft_consensus.cc:359] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.506047 17487 raft_consensus.cc:385] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.506333 17487 raft_consensus.cc:740] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.507014 17487 consensus_queue.cc:260] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.508834 17487 ts_tablet_manager.cc:1434] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:15:15.509637 17487 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.509799 17489 raft_consensus.cc:493] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.510275 17489 raft_consensus.cc:515] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.512197 17489 leader_election.cc:290] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.514348 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.514958 17292 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 0.
I20260709 12:15:15.515921 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.516137 17331 leader_election.cc:304] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.516659 17487 tablet_bootstrap.cc:654] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.516511 17441 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 0.
I20260709 12:15:15.522078 17487 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.522439 17487 ts_tablet_manager.cc:1403] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:15:15.523078 17489 raft_consensus.cc:2804] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.523443 17489 raft_consensus.cc:493] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.495146 17483 raft_consensus.cc:385] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.523752 17489 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.524235 17483 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.525230 17483 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.530316 17489 raft_consensus.cc:515] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.532429 17489 leader_election.cc:290] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.531021 17188 catalog_manager.cc:5697] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.533066 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:15.537698 17292 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.533689 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.539507 17441 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.534641 17487 raft_consensus.cc:359] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.541002 17487 raft_consensus.cc:385] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.541314 17487 raft_consensus.cc:740] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.535597 17484 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.542413 17487 consensus_queue.cc:260] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.544026 17484 ts_tablet_manager.cc:1434] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 12:15:15.541491 17495 raft_consensus.cc:493] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.537349 17489 raft_consensus.cc:493] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.536756 17483 ts_tablet_manager.cc:1434] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:15:15.544816 17495 raft_consensus.cc:515] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.548082 17484 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Bootstrap starting.
I20260709 12:15:15.548748 17489 raft_consensus.cc:515] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.553782 17483 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.551193 17489 leader_election.cc:290] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.559000 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.563202 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.564005 17487 ts_tablet_manager.cc:1434] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.041s	user 0.017s	sys 0.000s
I20260709 12:15:15.564447 17440 raft_consensus.cc:2468] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 0.
I20260709 12:15:15.564800 17495 leader_election.cc:290] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:15.565168 17487 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: Bootstrap starting.
I20260709 12:15:15.565292 17292 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 1.
I20260709 12:15:15.567502 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.568107 17367 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 0.
I20260709 12:15:15.569417 17405 leader_election.cc:304] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.570326 17331 leader_election.cc:304] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.571147 17489 raft_consensus.cc:2804] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.571776 17495 raft_consensus.cc:2804] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.572136 17495 raft_consensus.cc:493] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.572206 17489 raft_consensus.cc:697] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:15.572522 17495 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:15.571208 17331 leader_election.cc:343] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.574630 17488 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.575169 17488 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.575696 17441 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 1.
I20260709 12:15:15.577126 17331 leader_election.cc:304] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:15.573063 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.581619 17488 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.579201 17292 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 0.
I20260709 12:15:15.583787 17520 raft_consensus.cc:2804] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.584226 17520 raft_consensus.cc:493] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.585145 17520 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.585639 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.587971 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.588127 17441 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.588558 17367 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.589563 17256 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:15.586385 17489 consensus_queue.cc:237] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.591552 17488 raft_consensus.cc:2804] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.592350 17488 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.593236 17488 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.595360 17495 raft_consensus.cc:515] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.599823 17484 tablet_bootstrap.cc:654] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.602705 17522 raft_consensus.cc:493] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.606771 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:15.607961 17495 leader_election.cc:290] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:15.608669 17292 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.605338 17522 raft_consensus.cc:515] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.609972 17514 raft_consensus.cc:493] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.610105 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.612004 17367 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.612396 17514 raft_consensus.cc:515] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.627964 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.628607 17441 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.626864 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.630450 17484 tablet_bootstrap.cc:492] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: No bootstrap required, opened a new log
I20260709 12:15:15.632304 17488 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.626899 17514 leader_election.cc:290] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.634658 17484 ts_tablet_manager.cc:1403] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.007s
I20260709 12:15:15.630956 17367 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 1.
I20260709 12:15:15.635794 17256 leader_election.cc:304] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:15.636605 17514 raft_consensus.cc:2804] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.637085 17514 raft_consensus.cc:493] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.637405 17405 leader_election.cc:304] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.637513 17514 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.645980 17484 raft_consensus.cc:359] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.646522 17484 raft_consensus.cc:385] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.646694 17484 raft_consensus.cc:740] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.647223 17484 consensus_queue.cc:260] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.638305 17483 tablet_bootstrap.cc:654] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.626897 17520 raft_consensus.cc:515] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.636375 17522 leader_election.cc:290] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:15.652422 17487 tablet_bootstrap.cc:654] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.651297 17495 raft_consensus.cc:2804] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.629526 17366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.633289 17292 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 1.
I20260709 12:15:15.654199 17495 raft_consensus.cc:697] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:15.656313 17495 consensus_queue.cc:237] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.653859 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.657599 17367 raft_consensus.cc:2468] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 0.
I20260709 12:15:15.658659 17366 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.664755 17514 raft_consensus.cc:515] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.665937 17405 leader_election.cc:304] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.666684 17522 raft_consensus.cc:2804] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:15:15.666879 17405 leader_election.cc:343] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.667038 17522 raft_consensus.cc:493] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.667415 17522 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.677582 17520 leader_election.cc:290] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.678236 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.679008 17441 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.698985 17441 raft_consensus.cc:2468] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 1.
I20260709 12:15:15.701027 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:15.694008 17514 leader_election.cc:290] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.693668 17488 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.704633 17331 leader_election.cc:304] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.705075 17522 raft_consensus.cc:515] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.708771 17484 ts_tablet_manager.cc:1434] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: Time spent starting tablet: real 0.073s	user 0.021s	sys 0.039s
W20260709 12:15:15.710013 17331 leader_election.cc:343] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.712721 17366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.713872 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.713331 17527 raft_consensus.cc:2804] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.716403 17487 tablet_bootstrap.cc:492] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: No bootstrap required, opened a new log
I20260709 12:15:15.717710 17487 ts_tablet_manager.cc:1403] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: Time spent bootstrapping tablet: real 0.153s	user 0.018s	sys 0.019s
I20260709 12:15:15.716336 17366 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.733734 17483 tablet_bootstrap.cc:492] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.718389 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.748291 17441 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.721105 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.751031 17188 catalog_manager.cc:5697] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 0 to 1, leader changed from <none> to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.740549 17186 catalog_manager.cc:5697] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.752245 17440 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.727979 17522 leader_election.cc:290] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:15.717558 17367 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.720852 17514 raft_consensus.cc:493] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.755729 17514 raft_consensus.cc:515] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.758189 17514 leader_election.cc:290] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.735060 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.759473 17365 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.760027 17364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.731739 17527 raft_consensus.cc:697] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:15.760720 17364 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.761319 17527 consensus_queue.cc:237] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.762794 17256 leader_election.cc:304] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.762877 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.763738 17514 raft_consensus.cc:2804] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.764282 17514 raft_consensus.cc:493] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.764654 17514 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.765892 17366 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.764802 17487 raft_consensus.cc:359] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.766860 17487 raft_consensus.cc:385] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.767175 17487 raft_consensus.cc:740] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.763741 17439 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.767853 17487 consensus_queue.cc:260] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.769546 17256 leader_election.cc:304] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.770598 17483 ts_tablet_manager.cc:1403] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.223s	user 0.012s	sys 0.017s
I20260709 12:15:15.770325 17488 raft_consensus.cc:2804] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.773218 17483 raft_consensus.cc:359] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.773851 17483 raft_consensus.cc:385] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.774117 17483 raft_consensus.cc:740] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.718667 17520 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.776115 17520 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.734779 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:15.778724 17487 ts_tablet_manager.cc:1434] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
W20260709 12:15:15.779305 17405 leader_election.cc:343] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:15.781703 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360" is_pre_election: true
I20260709 12:15:15.785861 17520 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.787469 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.787459 17483 consensus_queue.cc:260] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.788158 17439 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 0.
I20260709 12:15:15.789367 17331 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.789646 17483 ts_tablet_manager.cc:1434] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:15:15.790472 17483 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: Bootstrap starting.
I20260709 12:15:15.786886 17514 raft_consensus.cc:515] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.782316 17291 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 0.
I20260709 12:15:15.791527 17488 raft_consensus.cc:697] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:15.796880 17532 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.795352 17365 raft_consensus.cc:2468] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 1.
I20260709 12:15:15.796087 17367 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.799355 17256 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.799505 17441 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.794975 17520 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.801620 17405 leader_election.cc:304] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:15.802034 17520 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.802372 17520 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.802484 17522 raft_consensus.cc:2804] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.803033 17522 raft_consensus.cc:697] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:15.803897 17522 consensus_queue.cc:237] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.800388 17488 consensus_queue.cc:237] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.806888 17531 raft_consensus.cc:2804] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.801528 17440 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.838183 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.839131 17365 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.826259 17514 leader_election.cc:290] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.839035 17186 catalog_manager.cc:5697] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.802877 17532 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.851584 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.852269 17440 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.859421 17532 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.860504 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:15.861706 17440 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.862221 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:15.862830 17441 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 0.
I20260709 12:15:15.865131 17483 tablet_bootstrap.cc:654] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:15.862774 17256 leader_election.cc:304] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:15.870651 17256 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:15.868239 17531 raft_consensus.cc:697] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:15.871495 17514 raft_consensus.cc:2804] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.872007 17514 raft_consensus.cc:697] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:15.871965 17531 consensus_queue.cc:237] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.872539 17532 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:15.873164 17532 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.873575 17532 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.875841 17514 consensus_queue.cc:237] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.876536 17186 catalog_manager.cc:5697] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:15.871289 17331 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:36889, user_credentials={real_user=slave}} blocked reactor thread for 59376us
I20260709 12:15:15.847847 17520 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.888775 17187 catalog_manager.cc:5697] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 0 to 1, leader changed from <none> to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.890240 17520 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.891665 17367 raft_consensus.cc:2393] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35f65a8690b24bc3a884546c776a3360 in current term 1: Already voted for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in this term.
I20260709 12:15:15.869545 17365 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 1.
I20260709 12:15:15.894104 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.894698 17441 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.895939 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:15.896634 17532 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.898061 17291 raft_consensus.cc:2393] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in current term 1: Already voted for candidate 35f65a8690b24bc3a884546c776a3360 in this term.
I20260709 12:15:15.901476 17441 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 1.
I20260709 12:15:15.902312 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:15.903156 17365 raft_consensus.cc:2393] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 35f65a8690b24bc3a884546c776a3360 in current term 1: Already voted for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in this term.
I20260709 12:15:15.904309 17483 tablet_bootstrap.cc:492] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: No bootstrap required, opened a new log
I20260709 12:15:15.904718 17483 ts_tablet_manager.cc:1403] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: Time spent bootstrapping tablet: real 0.114s	user 0.009s	sys 0.002s
I20260709 12:15:15.904855 17331 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:15.905889 17520 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:15.906400 17520 raft_consensus.cc:697] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:15.907267 17520 consensus_queue.cc:237] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.907610 17483 raft_consensus.cc:359] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.908226 17483 raft_consensus.cc:385] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:15.908515 17483 raft_consensus.cc:740] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Initialized, Role: FOLLOWER
I20260709 12:15:15.909129 17483 consensus_queue.cc:260] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:15.909855 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:15.910904 17441 raft_consensus.cc:2393] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 35f65a8690b24bc3a884546c776a3360 in current term 1: Already voted for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in this term.
I20260709 12:15:15.912688 17256 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360; no voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:15.911181 17532 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:15.913524 17531 raft_consensus.cc:2749] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:15.911139 17483 ts_tablet_manager.cc:1434] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 12:15:15.919951 17441 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:15.920306 17186 catalog_manager.cc:5697] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.920626 17187 catalog_manager.cc:5697] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.923066 17186 catalog_manager.cc:5697] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.929032 17532 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:15.952021 17532 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:15.989601 17441 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
mode: GRACEFUL
new_leader_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:56808
I20260709 12:15:15.990183 17441 raft_consensus.cc:606] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Received request to transfer leadership to TS 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:16.007225 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:53912
I20260709 12:15:16.007885 17365 raft_consensus.cc:606] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Received request to transfer leadership to TS 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:16.029402 17291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:37334
I20260709 12:15:16.029915 17291 raft_consensus.cc:606] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:16.037650 17291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbb00b52652b49589deaffba5311f162"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:37334
I20260709 12:15:16.038142 17291 raft_consensus.cc:606] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:16.043363 17441 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0494158f31647c99836512eef3c5da8"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:56808
I20260709 12:15:16.043969 17441 raft_consensus.cc:606] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:16.048908 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:53912
I20260709 12:15:16.049467 17365 raft_consensus.cc:606] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Received request to transfer leadership to TS eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:16.083693 17522 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.095448 17495 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.110178 17495 raft_consensus.cc:993] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: : Instructing follower 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:16.110642 17495 raft_consensus.cc:1081] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Signalling peer 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:16.112407 17291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:37330
I20260709 12:15:16.113096 17291 raft_consensus.cc:493] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.113601 17291 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.121289 17532 raft_consensus.cc:493] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.120988 17291 raft_consensus.cc:515] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.121747 17532 raft_consensus.cc:515] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.123181 17291 leader_election.cc:290] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.123409 17532 leader_election.cc:290] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.125052 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:16.125200 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:16.125464 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:16.125591 17441 raft_consensus.cc:3055] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.125908 17441 raft_consensus.cc:740] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:16.126017 17440 raft_consensus.cc:2393] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35f65a8690b24bc3a884546c776a3360 in current term 1: Already voted for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in this term.
I20260709 12:15:16.125195 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:16.127007 17365 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.126754 17441 consensus_queue.cc:260] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.128141 17441 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.132678 17256 leader_election.cc:304] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360; no voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:16.133496 17365 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 2.
I20260709 12:15:16.133845 17532 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:16.134519 17441 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 2.
I20260709 12:15:16.134591 17256 leader_election.cc:304] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:16.135388 17514 raft_consensus.cc:2804] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.135805 17514 raft_consensus.cc:697] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:16.136626 17514 consensus_queue.cc:237] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.141192 17532 raft_consensus.cc:2749] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:16.142940 17187 catalog_manager.cc:5697] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 1 to 2, leader changed from eecf17b6d17242b3bef574c2790ff838 (127.16.193.67) to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:16.145916 17532 raft_consensus.cc:493] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.146333 17532 raft_consensus.cc:515] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.148568 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" is_pre_election: true
I20260709 12:15:16.149281 17365 raft_consensus.cc:2393] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 35f65a8690b24bc3a884546c776a3360 in current term 1: Already voted for candidate eecf17b6d17242b3bef574c2790ff838 in this term.
I20260709 12:15:16.149313 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eecf17b6d17242b3bef574c2790ff838" is_pre_election: true
I20260709 12:15:16.150656 17532 leader_election.cc:290] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.152021 17256 leader_election.cc:304] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360; no voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:16.152657 17532 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:16.159034 17532 raft_consensus.cc:2749] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:16.161173 17520 consensus_queue.cc:1048] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.167352 17527 raft_consensus.cc:993] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:16.167764 17527 raft_consensus.cc:1081] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Signalling peer eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:16.170598 17440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:56776
I20260709 12:15:16.174436 17440 raft_consensus.cc:493] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.174808 17440 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.177558 17527 consensus_queue.cc:1048] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.181337 17440 raft_consensus.cc:515] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.188593 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:16.189162 17291 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.190974 17440 leader_election.cc:290] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:16.191965 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c36a4b0893c5471ca03b615379c68d82" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:16.192538 17365 raft_consensus.cc:3055] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.192865 17365 raft_consensus.cc:740] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:16.193638 17365 consensus_queue.cc:260] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.194888 17365 raft_consensus.cc:3060] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.196156 17291 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 2.
I20260709 12:15:16.197307 17527 consensus_queue.cc:1048] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.197320 17405 leader_election.cc:304] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:16.198396 17522 raft_consensus.cc:2804] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.198871 17522 raft_consensus.cc:697] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 2 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:16.199673 17522 consensus_queue.cc:237] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.207557 17186 catalog_manager.cc:5697] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 1 to 2, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:16.215929 17365 raft_consensus.cc:2468] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 2.
I20260709 12:15:16.226032 17520 consensus_queue.cc:1048] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.255155 17532 consensus_queue.cc:1048] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.264978 17514 consensus_queue.cc:1048] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.313741 17514 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.324641 17520 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.324932 17532 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.344830 17489 raft_consensus.cc:993] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:16.345353 17489 raft_consensus.cc:1081] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Signalling peer 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:16.346541 17514 raft_consensus.cc:993] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.346989 17514 raft_consensus.cc:1081] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.347210 17292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:37318
I20260709 12:15:16.347816 17292 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.348182 17292 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.348845 17367 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53904
I20260709 12:15:16.349395 17367 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.349756 17367 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.357432 17367 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.358048 17489 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.359790 17367 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.360402 17292 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.361827 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:16.362414 17441 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.362525 17292 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.363507 17291 raft_consensus.cc:1240] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Rejecting Update request from peer f52a92ccd3ba4e3f96cf73e6a8a92b56 for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:16.365605 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:16.368520 17489 consensus_queue.cc:1059] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.369722 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:16.371025 17292 raft_consensus.cc:3055] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.371449 17292 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:16.370471 17367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:16.372670 17292 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.373992 17292 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.381157 17441 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:16.383445 17367 raft_consensus.cc:3055] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.384320 17331 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:16.386886 17522 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:15:16.389770 17489 raft_consensus.cc:2804] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.390590 17489 raft_consensus.cc:697] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:16.383895 17367 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:16.392750 17367 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.398039 17367 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.391554 17489 consensus_queue.cc:237] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.396023 17514 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.404162 17292 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:16.407943 17367 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 2.
W20260709 12:15:16.408618 17520 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:15:16.414705 17256 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:16.416795 17551 raft_consensus.cc:993] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.417268 17551 raft_consensus.cc:1081] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.417387 17514 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.417893 17514 raft_consensus.cc:697] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:16.418696 17514 consensus_queue.cc:237] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.421581 17531 raft_consensus.cc:993] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.422005 17531 raft_consensus.cc:1081] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:16.424177 17551 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.426397 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0494158f31647c99836512eef3c5da8"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53888
I20260709 12:15:16.426980 17365 raft_consensus.cc:493] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.427098 17186 catalog_manager.cc:5697] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 1 to 2, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:16.427543 17365 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.429973 17364 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbb00b52652b49589deaffba5311f162"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53904
I20260709 12:15:16.430553 17364 raft_consensus.cc:493] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:16.430893 17364 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.435936 17532 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.438515 17365 raft_consensus.cc:515] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.442801 17366 raft_consensus.cc:1240] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Rejecting Update request from peer eecf17b6d17242b3bef574c2790ff838 for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:16.446887 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:16.448938 17551 consensus_queue.cc:1059] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.448495 17186 catalog_manager.cc:5697] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 1 to 2, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:16.449939 17495 raft_consensus.cc:3055] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.450279 17495 raft_consensus.cc:740] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:16.451485 17365 leader_election.cc:290] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.451095 17495 consensus_queue.cc:260] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.452955 17495 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.453049 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
W20260709 12:15:16.454156 17522 consensus_queue.cc:1175] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "35f65a8690b24bc3a884546c776a3360" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:15:16.462332 17364 raft_consensus.cc:515] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.464545 17367 raft_consensus.cc:1240] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Rejecting Update request from peer 35f65a8690b24bc3a884546c776a3360 for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:16.468048 17440 raft_consensus.cc:2468] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:16.470577 17364 leader_election.cc:290] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:16.471904 17514 consensus_queue.cc:1059] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:16.472524 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:16.472898 17532 raft_consensus.cc:3055] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:16.473281 17532 raft_consensus.cc:740] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:16.473382 17331 leader_election.cc:304] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:16.474136 17532 consensus_queue.cc:260] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.475275 17532 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.475133 17440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:16.476425 17440 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.477741 17489 raft_consensus.cc:2804] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.478960 17489 raft_consensus.cc:697] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:16.479784 17489 consensus_queue.cc:237] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.484558 17440 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:16.485978 17331 leader_election.cc:304] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:16.488663 17292 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:16.486784 17489 raft_consensus.cc:2804] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:16.488978 17187 catalog_manager.cc:5697] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 1 to 2, leader changed from eecf17b6d17242b3bef574c2790ff838 (127.16.193.67) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:16.489775 17489 raft_consensus.cc:697] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:16.492519 17489 consensus_queue.cc:237] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:16.502820 17187 catalog_manager.cc:5697] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 1 to 2, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:16.540946 17364 raft_consensus.cc:1275] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.542173 17532 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.550158 17440 raft_consensus.cc:1275] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.551940 17532 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.652438 17292 raft_consensus.cc:1275] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.653620 17495 consensus_queue.cc:1048] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.662740 17364 raft_consensus.cc:1275] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.664981 17551 consensus_queue.cc:1048] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.833609 17292 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.834836 17489 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.843516 17440 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.844981 17527 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.907862 17440 raft_consensus.cc:1275] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.909050 17520 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.921348 17292 raft_consensus.cc:1275] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.921516 17291 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.931355 17291 raft_consensus.cc:1275] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.933070 17520 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:16.934821 17520 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.945611 17440 raft_consensus.cc:1275] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.948491 17520 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.965703 17364 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.967147 17532 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:16.975600 17440 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:16.983542 17440 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:16.984789 17532 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:19.078433 17157 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:15:19.079305 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:19.095111 17292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:15:19.095578 17292 raft_consensus.cc:606] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Received request to transfer leadership to TS eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:19.105223 17364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:19.105636 17364 raft_consensus.cc:606] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Received request to transfer leadership to TS eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:19.106707 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:19.107158 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:19.194626 17527 raft_consensus.cc:993] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:19.195091 17571 raft_consensus.cc:1081] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Signalling peer eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:19.196455 17440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:56776
I20260709 12:15:19.196913 17440 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:19.197202 17440 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.201571 17440 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.203042 17440 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:19.203795 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:19.204166 17364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:19.204399 17292 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.204710 17364 raft_consensus.cc:3055] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:19.205034 17364 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:19.205803 17364 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.206959 17364 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.210914 17292 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:19.212157 17405 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:19.212915 17551 raft_consensus.cc:2804] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:19.213485 17364 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:19.214146 17551 raft_consensus.cc:697] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:19.215039 17551 consensus_queue.cc:237] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.222175 17188 catalog_manager.cc:5697] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 2 to 3, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.490319 17532 raft_consensus.cc:993] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: : Instructing follower eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:19.490751 17578 raft_consensus.cc:1081] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Signalling peer eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:19.492144 17440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:56792
I20260709 12:15:19.492599 17440 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:19.492904 17440 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.498122 17440 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.499879 17440 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:19.501459 17292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:19.501886 17364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:19.502012 17292 raft_consensus.cc:3055] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:19.502415 17292 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:19.502425 17364 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.503288 17292 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.504431 17292 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:19.509614 17364 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:19.510850 17405 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:19.511090 17292 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:19.511644 17551 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:19.512933 17551 raft_consensus.cc:697] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:19.513521 17551 consensus_queue.cc:237] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:19.519589 17188 catalog_manager.cc:5697] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 2 to 3, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.729133 17364 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:19.730360 17551 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:19.738752 17292 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:19.740238 17579 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:19.936487 17292 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:19.937834 17579 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:19.946936 17364 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:19.948309 17572 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:20.108040 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:20.113034 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:20.113504 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:21.114427 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:21.119385 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:21.119796 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:22.149252 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:22.401090 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:22.401546 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:15:23.392083 17220 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 17160
W20260709 12:15:23.392966 17220 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 17223
W20260709 12:15:23.393823 17220 debug-util.cc:398] Leaking SignalData structure 0x7b0800194c60 after lost signal to thread 17316
W20260709 12:15:23.394637 17220 debug-util.cc:398] Leaking SignalData structure 0x7b08001cd820 after lost signal to thread 17391
W20260709 12:15:23.395807 17220 debug-util.cc:398] Leaking SignalData structure 0x7b0800159ca0 after lost signal to thread 17465
I20260709 12:15:23.403465 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:23.412016 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:23.412511 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:24.413597 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:24.418327 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:24.418757 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:25.419860 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:25.428009 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:25.428535 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:26.429389 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:26.434085 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:26.434396 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:27.435127 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:27.439683 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:27.440012 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:28.440889 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:28.445642 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:28.446030 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:29.472409 17157 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:15:29.476848 17291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "184227df1321443489980c96dbba4e5f"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:37334
I20260709 12:15:29.477535 17291 raft_consensus.cc:606] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:29.482138 17291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:37334
I20260709 12:15:29.482605 17291 raft_consensus.cc:606] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:29.487439 17291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:37334
I20260709 12:15:29.487908 17291 raft_consensus.cc:606] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:29.492321 17439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:56808
I20260709 12:15:29.492774 17439 raft_consensus.cc:606] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:29.496907 17439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
mode: GRACEFUL
new_leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:56808
I20260709 12:15:29.497331 17439 raft_consensus.cc:606] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Received request to transfer leadership to TS f52a92ccd3ba4e3f96cf73e6a8a92b56
I20260709 12:15:29.657006 17598 raft_consensus.cc:993] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.657573 17600 raft_consensus.cc:1081] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.659191 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53904
I20260709 12:15:29.659852 17365 raft_consensus.cc:493] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.660307 17365 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:29.665339 17365 raft_consensus.cc:515] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.666872 17365 leader_election.cc:290] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 3 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:29.667789 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:29.668222 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c16bb43a874fc19023a6aa38512b59" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:29.668496 17291 raft_consensus.cc:3055] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:29.668833 17291 raft_consensus.cc:740] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:29.668859 17439 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:29.669610 17291 consensus_queue.cc:260] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.670643 17291 raft_consensus.cc:3060] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:29.674072 17439 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 3.
I20260709 12:15:29.675510 17291 raft_consensus.cc:2468] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 3.
I20260709 12:15:29.675467 17331 leader_election.cc:304] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:29.676489 17595 raft_consensus.cc:2804] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:29.677335 17595 raft_consensus.cc:697] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:29.678257 17595 consensus_queue.cc:237] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.689086 17186 catalog_manager.cc:5697] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 2 to 3, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 3 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.754376 17599 raft_consensus.cc:993] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.754876 17607 raft_consensus.cc:1081] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.756219 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53888
I20260709 12:15:29.756660 17365 raft_consensus.cc:493] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.756942 17365 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.761021 17365 raft_consensus.cc:515] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.762362 17365 leader_election.cc:290] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 4 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:29.763100 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:29.763451 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de41642be634fb2af89f5788d4847aa" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:29.763655 17291 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.764008 17439 raft_consensus.cc:3055] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:29.764329 17439 raft_consensus.cc:740] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:29.765105 17439 consensus_queue.cc:260] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.766275 17439 raft_consensus.cc:3060] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.769524 17291 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 4.
I20260709 12:15:29.770514 17331 leader_election.cc:304] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:29.771268 17595 raft_consensus.cc:2804] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:29.771754 17595 raft_consensus.cc:697] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:29.772562 17439 raft_consensus.cc:2468] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 4.
I20260709 12:15:29.772526 17595 consensus_queue.cc:237] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.779726 17186 catalog_manager.cc:5697] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 3 to 4, leader changed from eecf17b6d17242b3bef574c2790ff838 (127.16.193.67) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 4 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.838248 17599 raft_consensus.cc:993] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.838714 17607 raft_consensus.cc:1081] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.840116 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53888
I20260709 12:15:29.840560 17365 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.840842 17365 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.844681 17365 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.846001 17365 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 4 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:29.846788 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:29.847069 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:29.847316 17291 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.847635 17439 raft_consensus.cc:3055] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:29.847903 17439 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:29.848582 17439 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.849525 17439 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:29.851589 17291 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 4.
I20260709 12:15:29.852474 17331 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:29.853093 17595 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:29.853559 17595 raft_consensus.cc:697] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:29.854285 17439 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 4.
I20260709 12:15:29.854423 17595 consensus_queue.cc:237] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.861200 17186 catalog_manager.cc:5697] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 3 to 4, leader changed from eecf17b6d17242b3bef574c2790ff838 (127.16.193.67) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 4 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.946012 17598 raft_consensus.cc:993] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.946427 17600 raft_consensus.cc:1081] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.947763 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "184227df1321443489980c96dbba4e5f"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53904
I20260709 12:15:29.948201 17365 raft_consensus.cc:493] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.948486 17365 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.952363 17365 raft_consensus.cc:515] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.953647 17365 leader_election.cc:290] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:29.954416 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:29.954713 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:29.954897 17291 raft_consensus.cc:3055] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.955273 17291 raft_consensus.cc:740] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:29.955281 17439 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.956007 17291 consensus_queue.cc:260] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.956822 17291 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.959867 17439 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:29.960474 17291 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:29.960817 17331 leader_election.cc:304] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:29.961514 17595 raft_consensus.cc:2804] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.961895 17595 raft_consensus.cc:697] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:29.962639 17595 consensus_queue.cc:237] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.968591 17186 catalog_manager.cc:5697] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 1 to 2, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.976900 17598 raft_consensus.cc:993] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: : Instructing follower f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.977358 17600 raft_consensus.cc:1081] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Signalling peer f52a92ccd3ba4e3f96cf73e6a8a92b56 to start an election
I20260709 12:15:29.978632 17365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
 from {username='slave'} at 127.0.0.1:53904
I20260709 12:15:29.979065 17365 raft_consensus.cc:493] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.979357 17365 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.983429 17365 raft_consensus.cc:515] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.984902 17365 leader_election.cc:290] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:29.985724 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:29.985858 17439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05ef912897f4cb2ac5a9c41ef27bc37" candidate_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:29.986291 17291 raft_consensus.cc:3055] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.986330 17439 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.986724 17291 raft_consensus.cc:740] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:29.987535 17291 consensus_queue.cc:260] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:29.988581 17291 raft_consensus.cc:3060] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.992985 17439 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:29.994207 17331 leader_election.cc:304] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eecf17b6d17242b3bef574c2790ff838, f52a92ccd3ba4e3f96cf73e6a8a92b56; no voters: 
I20260709 12:15:29.994355 17291 raft_consensus.cc:2468] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52a92ccd3ba4e3f96cf73e6a8a92b56 in term 2.
I20260709 12:15:29.994915 17595 raft_consensus.cc:2804] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.995412 17595 raft_consensus.cc:697] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Leader. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:29.996126 17595 consensus_queue.cc:237] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:30.002341 17186 catalog_manager.cc:5697] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 reported cstate change: term changed from 1 to 2, leader changed from 35f65a8690b24bc3a884546c776a3360 (127.16.193.65) to f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:30.078019 17291 raft_consensus.cc:1275] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:30.079103 17595 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:30.089090 17439 raft_consensus.cc:1275] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:30.091001 17595 consensus_queue.cc:1048] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:30.252770 17441 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:30.254009 17614 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:30.264319 17291 raft_consensus.cc:1275] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:30.264506 17441 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:30.265625 17614 consensus_queue.cc:1048] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:30.266217 17595 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:30.279296 17291 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:30.280709 17614 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:30.368691 17291 raft_consensus.cc:1275] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:30.369843 17614 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:30.379591 17441 raft_consensus.cc:1275] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:30.381012 17595 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:30.383628 17291 raft_consensus.cc:1275] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:30.385080 17595 consensus_queue.cc:1048] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:30.398327 17441 raft_consensus.cc:1275] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer f52a92ccd3ba4e3f96cf73e6a8a92b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:30.399602 17595 consensus_queue.cc:1048] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:32.623160 17157 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:15:32.624405 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:32.631187 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "184227df1321443489980c96dbba4e5f"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:32.631979 17365 raft_consensus.cc:606] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Received request to transfer leadership to TS 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:32.634354 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:32.634802 17365 raft_consensus.cc:606] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Received request to transfer leadership to TS eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:32.636818 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbb00b52652b49589deaffba5311f162"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:32.637233 17365 raft_consensus.cc:606] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Received request to transfer leadership to TS 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:32.639336 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0494158f31647c99836512eef3c5da8"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:32.639731 17365 raft_consensus.cc:606] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Received request to transfer leadership to TS eecf17b6d17242b3bef574c2790ff838
I20260709 12:15:32.641867 17365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5"
dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
mode: GRACEFUL
new_leader_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:15:32.642366 17365 raft_consensus.cc:606] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Received request to transfer leadership to TS 35f65a8690b24bc3a884546c776a3360
I20260709 12:15:32.643555 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:32.643932 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:32.787926 17623 raft_consensus.cc:993] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:32.788373 17623 raft_consensus.cc:1081] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Signalling peer eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:32.790022 17441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0494158f31647c99836512eef3c5da8"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:56776
I20260709 12:15:32.790690 17441 raft_consensus.cc:493] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.791061 17441 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.795934 17441 raft_consensus.cc:515] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.797825 17441 leader_election.cc:290] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:32.799127 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:32.799072 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0494158f31647c99836512eef3c5da8" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:32.799896 17365 raft_consensus.cc:3055] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:32.799973 17291 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.800170 17365 raft_consensus.cc:740] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:32.800844 17365 consensus_queue.cc:260] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.801723 17365 raft_consensus.cc:3060] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.804643 17291 raft_consensus.cc:2468] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:32.805799 17405 leader_election.cc:304] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:32.806191 17365 raft_consensus.cc:2468] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 3.
I20260709 12:15:32.806584 17607 raft_consensus.cc:2804] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:32.807659 17607 raft_consensus.cc:697] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:32.808607 17607 consensus_queue.cc:237] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.822763 17186 catalog_manager.cc:5697] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 2 to 3, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.835872 17627 raft_consensus.cc:993] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:32.836323 17623 raft_consensus.cc:1081] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Signalling peer 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:32.837617 17291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "184227df1321443489980c96dbba4e5f"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:37318
I20260709 12:15:32.838056 17291 raft_consensus.cc:493] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.838320 17291 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.842226 17291 raft_consensus.cc:515] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.844087 17291 leader_election.cc:290] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 3 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:32.845212 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:32.845363 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "184227df1321443489980c96dbba4e5f" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:32.845711 17365 raft_consensus.cc:3055] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:32.845867 17441 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.846065 17365 raft_consensus.cc:740] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:32.846761 17365 consensus_queue.cc:260] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.847586 17365 raft_consensus.cc:3060] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.850154 17441 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 3.
I20260709 12:15:32.851320 17256 leader_election.cc:304] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:32.851857 17365 raft_consensus.cc:2468] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 3.
I20260709 12:15:32.852900 17635 raft_consensus.cc:2804] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:32.853842 17635 raft_consensus.cc:697] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 3 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:32.854609 17635 consensus_queue.cc:237] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:32.860999 17186 catalog_manager.cc:5697] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 2 to 3, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:32.991690 17627 raft_consensus.cc:993] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:32.992199 17623 raft_consensus.cc:1081] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Signalling peer eecf17b6d17242b3bef574c2790ff838 to start an election
I20260709 12:15:32.993541 17441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66"
dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
 from {username='slave'} at 127.0.0.1:56776
I20260709 12:15:32.993922 17441 raft_consensus.cc:493] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.994139 17441 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:32.998682 17441 raft_consensus.cc:515] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.000245 17441 leader_election.cc:290] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 5 election: Requested vote from peers 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889), f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015)
I20260709 12:15:33.001099 17291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "35f65a8690b24bc3a884546c776a3360"
I20260709 12:15:33.001466 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86532393c7d645a587c7e9c076b75f66" candidate_uuid: "eecf17b6d17242b3bef574c2790ff838" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:33.001762 17291 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:33.001981 17365 raft_consensus.cc:3055] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:15:33.002278 17365 raft_consensus.cc:740] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:33.003021 17365 consensus_queue.cc:260] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.004422 17365 raft_consensus.cc:3060] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:33.007841 17291 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 5.
I20260709 12:15:33.008929 17405 leader_election.cc:304] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:33.009680 17607 raft_consensus.cc:2804] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:15:33.010160 17607 raft_consensus.cc:697] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 5 LEADER]: Becoming Leader. State: Replica: eecf17b6d17242b3bef574c2790ff838, State: Running, Role: LEADER
I20260709 12:15:33.010303 17365 raft_consensus.cc:2468] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eecf17b6d17242b3bef574c2790ff838 in term 5.
I20260709 12:15:33.011107 17607 consensus_queue.cc:237] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.018010 17186 catalog_manager.cc:5697] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 reported cstate change: term changed from 4 to 5, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to eecf17b6d17242b3bef574c2790ff838 (127.16.193.67). New cstate: current_term: 5 leader_uuid: "eecf17b6d17242b3bef574c2790ff838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.082832 17623 raft_consensus.cc:993] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:33.083284 17627 raft_consensus.cc:1081] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Signalling peer 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:33.084699 17291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbb00b52652b49589deaffba5311f162"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:37318
I20260709 12:15:33.085150 17291 raft_consensus.cc:493] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.085424 17291 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:33.089226 17291 raft_consensus.cc:515] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.090592 17291 leader_election.cc:290] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 3 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:33.091508 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:33.091722 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb00b52652b49589deaffba5311f162" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:33.092056 17365 raft_consensus.cc:3055] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:33.092196 17441 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:33.092401 17365 raft_consensus.cc:740] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:33.093245 17365 consensus_queue.cc:260] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.094357 17365 raft_consensus.cc:3060] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:33.098356 17441 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 3.
I20260709 12:15:33.099385 17256 leader_election.cc:304] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:33.100040 17635 raft_consensus.cc:2804] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:33.100210 17365 raft_consensus.cc:2468] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 3.
I20260709 12:15:33.100446 17635 raft_consensus.cc:697] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 3 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:33.101382 17635 consensus_queue.cc:237] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.108217 17186 catalog_manager.cc:5697] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 2 to 3, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.176700 17623 raft_consensus.cc:993] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: : Instructing follower 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:33.177134 17627 raft_consensus.cc:1081] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Signalling peer 35f65a8690b24bc3a884546c776a3360 to start an election
I20260709 12:15:33.178396 17291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5"
dest_uuid: "35f65a8690b24bc3a884546c776a3360"
 from {username='slave'} at 127.0.0.1:37318
I20260709 12:15:33.178756 17291 raft_consensus.cc:493] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.178982 17291 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.183095 17291 raft_consensus.cc:515] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.184773 17291 leader_election.cc:290] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Requested vote from peers f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015), eecf17b6d17242b3bef574c2790ff838 (127.16.193.67:41171)
I20260709 12:15:33.185318 17365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56"
I20260709 12:15:33.185626 17441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b6545b38e84924b860ddb9d791bcb5" candidate_uuid: "35f65a8690b24bc3a884546c776a3360" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eecf17b6d17242b3bef574c2790ff838"
I20260709 12:15:33.185849 17365 raft_consensus.cc:3055] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:33.186141 17441 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.186141 17365 raft_consensus.cc:740] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f52a92ccd3ba4e3f96cf73e6a8a92b56, State: Running, Role: LEADER
I20260709 12:15:33.187016 17365 consensus_queue.cc:260] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.188092 17365 raft_consensus.cc:3060] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.192055 17441 raft_consensus.cc:2468] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 2.
I20260709 12:15:33.192993 17256 leader_election.cc:304] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f65a8690b24bc3a884546c776a3360, eecf17b6d17242b3bef574c2790ff838; no voters: 
I20260709 12:15:33.193567 17365 raft_consensus.cc:2468] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f65a8690b24bc3a884546c776a3360 in term 2.
I20260709 12:15:33.193608 17635 raft_consensus.cc:2804] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:33.194047 17635 raft_consensus.cc:697] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Becoming Leader. State: Replica: 35f65a8690b24bc3a884546c776a3360, State: Running, Role: LEADER
I20260709 12:15:33.194936 17635 consensus_queue.cc:237] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } }
I20260709 12:15:33.201562 17186 catalog_manager.cc:5697] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 reported cstate change: term changed from 1 to 2, leader changed from f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66) to 35f65a8690b24bc3a884546c776a3360 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "35f65a8690b24bc3a884546c776a3360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.205852 17291 raft_consensus.cc:1275] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.207501 17607 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.222422 17365 raft_consensus.cc:1275] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.223692 17629 consensus_queue.cc:1048] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:33.411748 17365 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 5 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:15:33.412954 17629 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:33.423491 17291 raft_consensus.cc:1275] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 5 FOLLOWER]: Refusing update from remote peer eecf17b6d17242b3bef574c2790ff838: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:15:33.424854 17643 consensus_queue.cc:1048] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f65a8690b24bc3a884546c776a3360" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:15:33.432904 17365 raft_consensus.cc:1275] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.433990 17635 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.442677 17441 raft_consensus.cc:1275] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:15:33.443317 17388 debug-util.cc:398] Leaking SignalData structure 0x7b080012d060 after lost signal to thread 17160
W20260709 12:15:33.444366 17388 debug-util.cc:398] Leaking SignalData structure 0x7b080012d740 after lost signal to thread 17223
I20260709 12:15:33.444274 17635 consensus_queue.cc:1048] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:15:33.445739 17388 debug-util.cc:398] Leaking SignalData structure 0x7b0800117a20 after lost signal to thread 17316
W20260709 12:15:33.448019 17388 debug-util.cc:398] Leaking SignalData structure 0x7b08001a74e0 after lost signal to thread 17391
W20260709 12:15:33.449194 17388 debug-util.cc:398] Leaking SignalData structure 0x7b080010f340 after lost signal to thread 17465
I20260709 12:15:33.560595 17365 raft_consensus.cc:1275] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.561278 17441 raft_consensus.cc:1275] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.561935 17635 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.563438 17635 consensus_queue.cc:1048] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.589607 17441 raft_consensus.cc:1275] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:33.591423 17635 consensus_queue.cc:1048] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eecf17b6d17242b3bef574c2790ff838" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:33.605391 17365 raft_consensus.cc:1275] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Refusing update from remote peer 35f65a8690b24bc3a884546c776a3360: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:33.607190 17635 consensus_queue.cc:1048] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f52a92ccd3ba4e3f96cf73e6a8a92b56" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:33.645176 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:33.653373 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:33.653959 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:34.655315 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:34.663518 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:34.664041 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:35.665243 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:35.673727 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:35.674257 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:36.675367 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:36.680298 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:36.680713 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:37.681779 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:37.687853 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:37.688275 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:38.689210 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:38.695351 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:38.695696 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:39.696557 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:39.701447 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:39.701861 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:40.702881 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.707767 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:40.708175 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:41.709100 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:41.714062 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:41.714443 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:42.742527 17157 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:15:42.743153 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
I20260709 12:15:42.774857 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:42.775671 17157 tablet_replica.cc:333] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.776515 17157 raft_consensus.cc:2243] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.777170 17157 raft_consensus.cc:2272] T 8de41642be634fb2af89f5788d4847aa P 35f65a8690b24bc3a884546c776a3360 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.779626 17157 tablet_replica.cc:333] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.780118 17157 raft_consensus.cc:2243] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.780627 17157 raft_consensus.cc:2272] T 86532393c7d645a587c7e9c076b75f66 P 35f65a8690b24bc3a884546c776a3360 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.782390 17157 tablet_replica.cc:333] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.782980 17157 raft_consensus.cc:2243] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.783841 17157 raft_consensus.cc:2272] T d7b6545b38e84924b860ddb9d791bcb5 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.785392 17157 tablet_replica.cc:333] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.785858 17157 raft_consensus.cc:2243] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.786756 17157 raft_consensus.cc:2272] T bbb00b52652b49589deaffba5311f162 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.788367 17157 tablet_replica.cc:333] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.788900 17157 raft_consensus.cc:2243] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.789400 17157 raft_consensus.cc:2272] T 70c16bb43a874fc19023a6aa38512b59 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.791067 17157 tablet_replica.cc:333] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.791548 17157 raft_consensus.cc:2243] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.791999 17157 raft_consensus.cc:2272] T c36a4b0893c5471ca03b615379c68d82 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.793547 17157 tablet_replica.cc:333] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.794015 17157 raft_consensus.cc:2243] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.794482 17157 raft_consensus.cc:2272] T c05ef912897f4cb2ac5a9c41ef27bc37 P 35f65a8690b24bc3a884546c776a3360 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.796129 17157 tablet_replica.cc:333] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.796568 17157 raft_consensus.cc:2243] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.797016 17157 raft_consensus.cc:2272] T d0494158f31647c99836512eef3c5da8 P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.798635 17157 tablet_replica.cc:333] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360: stopping tablet replica
I20260709 12:15:42.799032 17157 raft_consensus.cc:2243] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.799863 17157 raft_consensus.cc:2272] T 184227df1321443489980c96dbba4e5f P 35f65a8690b24bc3a884546c776a3360 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:42.906410 17405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:36889: connect: Connection refused (error 111)
I20260709 12:15:42.908654 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
W20260709 12:15:42.910429 17405 consensus_peers.cc:597] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 -> Peer 35f65a8690b24bc3a884546c776a3360 (127.16.193.65:36889): Couldn't send request to peer 35f65a8690b24bc3a884546c776a3360. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:36889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:42.927053 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:15:42.952134 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:42.952929 17157 tablet_replica.cc:333] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.953997 17157 raft_consensus.cc:2243] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.955106 17157 raft_consensus.cc:2272] T c05ef912897f4cb2ac5a9c41ef27bc37 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.956934 17157 tablet_replica.cc:333] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.957373 17157 raft_consensus.cc:2243] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.957813 17157 raft_consensus.cc:2272] T c36a4b0893c5471ca03b615379c68d82 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.959532 17157 tablet_replica.cc:333] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.959991 17157 raft_consensus.cc:2243] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.960678 17157 raft_consensus.cc:2272] T 8de41642be634fb2af89f5788d4847aa P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.962177 17157 tablet_replica.cc:333] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.962596 17157 raft_consensus.cc:2243] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.962953 17157 raft_consensus.cc:2272] T 86532393c7d645a587c7e9c076b75f66 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.964598 17157 tablet_replica.cc:333] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.965029 17157 raft_consensus.cc:2243] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.965482 17157 raft_consensus.cc:2272] T bbb00b52652b49589deaffba5311f162 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.966830 17157 tablet_replica.cc:333] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.967344 17157 raft_consensus.cc:2243] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.968187 17157 raft_consensus.cc:2272] T 70c16bb43a874fc19023a6aa38512b59 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.969746 17157 tablet_replica.cc:333] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.970134 17157 raft_consensus.cc:2243] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.970547 17157 raft_consensus.cc:2272] T d7b6545b38e84924b860ddb9d791bcb5 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.972069 17157 tablet_replica.cc:333] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.972465 17157 raft_consensus.cc:2243] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.972846 17157 raft_consensus.cc:2272] T d0494158f31647c99836512eef3c5da8 P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.974284 17157 tablet_replica.cc:333] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56: stopping tablet replica
I20260709 12:15:42.974674 17157 raft_consensus.cc:2243] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.975037 17157 raft_consensus.cc:2272] T 184227df1321443489980c96dbba4e5f P f52a92ccd3ba4e3f96cf73e6a8a92b56 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:42.999078 17405 consensus_peers.cc:597] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 -> Peer f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015): Couldn't send request to peer f52a92ccd3ba4e3f96cf73e6a8a92b56. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:33015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.033641 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:15:43.038036 17405 consensus_peers.cc:597] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 -> Peer f52a92ccd3ba4e3f96cf73e6a8a92b56 (127.16.193.66:33015): Couldn't send request to peer f52a92ccd3ba4e3f96cf73e6a8a92b56. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:33015: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.051779 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:15:43.084786 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:43.085546 17157 tablet_replica.cc:333] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.086483 17157 raft_consensus.cc:2243] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.087152 17157 raft_consensus.cc:2272] T 184227df1321443489980c96dbba4e5f P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.089320 17157 tablet_replica.cc:333] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.089843 17157 raft_consensus.cc:2243] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.090622 17157 raft_consensus.cc:2272] T d0494158f31647c99836512eef3c5da8 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.092343 17157 tablet_replica.cc:333] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.092804 17157 raft_consensus.cc:2243] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.093195 17157 raft_consensus.cc:2272] T 8de41642be634fb2af89f5788d4847aa P eecf17b6d17242b3bef574c2790ff838 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.094830 17157 tablet_replica.cc:333] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.095299 17157 raft_consensus.cc:2243] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.096040 17157 raft_consensus.cc:2272] T 86532393c7d645a587c7e9c076b75f66 P eecf17b6d17242b3bef574c2790ff838 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.097615 17157 tablet_replica.cc:333] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.098081 17157 raft_consensus.cc:2243] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.098767 17157 raft_consensus.cc:2272] T c36a4b0893c5471ca03b615379c68d82 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.100416 17157 tablet_replica.cc:333] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.100873 17157 raft_consensus.cc:2243] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.101323 17157 raft_consensus.cc:2272] T c05ef912897f4cb2ac5a9c41ef27bc37 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.102762 17157 tablet_replica.cc:333] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.103196 17157 raft_consensus.cc:2243] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.103688 17157 raft_consensus.cc:2272] T bbb00b52652b49589deaffba5311f162 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.105082 17157 tablet_replica.cc:333] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.105513 17157 raft_consensus.cc:2243] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.105878 17157 raft_consensus.cc:2272] T d7b6545b38e84924b860ddb9d791bcb5 P eecf17b6d17242b3bef574c2790ff838 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.107357 17157 tablet_replica.cc:333] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838: stopping tablet replica
I20260709 12:15:43.107758 17157 raft_consensus.cc:2243] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.108194 17157 raft_consensus.cc:2272] T 70c16bb43a874fc19023a6aa38512b59 P eecf17b6d17242b3bef574c2790ff838 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.134912 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:15:43.152135 17157 master.cc:562] Master@127.16.193.126:45553 shutting down...
I20260709 12:15:43.181828 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.182435 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.182758 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1561d7861311457ba94c83ec0f7ab08d: stopping tablet replica
I20260709 12:15:43.217787 17157 master.cc:584] Master@127.16.193.126:45553 shutdown complete.
35f65a8690b24bc3a884546c776a3360  4
eecf17b6d17242b3bef574c2790ff838  1
f52a92ccd3ba4e3f96cf73e6a8a92b56  4
35f65a8690b24bc3a884546c776a3360  3
eecf17b6d17242b3bef574c2790ff838  3
f52a92ccd3ba4e3f96cf73e6a8a92b56  3
35f65a8690b24bc3a884546c776a3360  0
eecf17b6d17242b3bef574c2790ff838  1
f52a92ccd3ba4e3f96cf73e6a8a92b56  8
35f65a8690b24bc3a884546c776a3360  3
eecf17b6d17242b3bef574c2790ff838  3
f52a92ccd3ba4e3f96cf73e6a8a92b56  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (30749 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:15:43.258661 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:44857
I20260709 12:15:43.259835 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.265519 17663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.266209 17664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.266958 17666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.267586 17157 server_base.cc:1061] running on GCE node
I20260709 12:15:43.268500 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.268693 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.268864 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599343268834 us; error 0 us; skew 500 ppm
I20260709 12:15:43.269382 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.272069 17157 webserver.cc:533] Webserver started at http://127.16.193.126:42185/ using document root <none> and password file <none>
I20260709 12:15:43.272572 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.272768 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.273054 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.274129 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "00e02b5cf0e34d3d9522cccdc716b616"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.278344 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:15:43.281658 17671 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.282336 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:15:43.282615 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "00e02b5cf0e34d3d9522cccdc716b616"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.282945 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.305011 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.306208 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.343420 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:44857
I20260709 12:15:43.343514 17722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:44857 every 8 connection(s)
I20260709 12:15:43.347653 17723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.356690 17723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616: Bootstrap starting.
I20260709 12:15:43.361620 17723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:43.365410 17723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616: No bootstrap required, opened a new log
I20260709 12:15:43.367460 17723 raft_consensus.cc:359] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER }
I20260709 12:15:43.367858 17723 raft_consensus.cc:385] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:43.368031 17723 raft_consensus.cc:740] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00e02b5cf0e34d3d9522cccdc716b616, State: Initialized, Role: FOLLOWER
I20260709 12:15:43.368494 17723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER }
I20260709 12:15:43.368889 17723 raft_consensus.cc:399] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:43.369068 17723 raft_consensus.cc:493] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:43.369274 17723 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:43.372922 17723 raft_consensus.cc:515] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER }
I20260709 12:15:43.373368 17723 leader_election.cc:304] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00e02b5cf0e34d3d9522cccdc716b616; no voters: 
I20260709 12:15:43.374393 17723 leader_election.cc:290] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:43.374754 17726 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:43.376102 17726 raft_consensus.cc:697] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 1 LEADER]: Becoming Leader. State: Replica: 00e02b5cf0e34d3d9522cccdc716b616, State: Running, Role: LEADER
I20260709 12:15:43.376804 17726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER }
I20260709 12:15:43.377247 17723 sys_catalog.cc:565] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:43.381804 17727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00e02b5cf0e34d3d9522cccdc716b616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER } }
I20260709 12:15:43.382529 17727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:43.382346 17728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00e02b5cf0e34d3d9522cccdc716b616. Latest consensus state: current_term: 1 leader_uuid: "00e02b5cf0e34d3d9522cccdc716b616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e02b5cf0e34d3d9522cccdc716b616" member_type: VOTER } }
I20260709 12:15:43.383054 17728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:43.386000 17734 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:43.391175 17734 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:43.393038 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:43.399662 17734 catalog_manager.cc:1383] Generated new cluster ID: 8f774b363c9f42b0b26c4f320441a128
I20260709 12:15:43.399909 17734 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:43.415211 17734 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:43.416373 17734 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:43.433420 17734 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616: Generated new TSK 0
I20260709 12:15:43.433995 17734 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:43.459069 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.464762 17744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.465766 17745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.467056 17747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.467274 17157 server_base.cc:1061] running on GCE node
I20260709 12:15:43.468238 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.468410 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.468526 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599343468516 us; error 0 us; skew 500 ppm
I20260709 12:15:43.468971 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.471019 17157 webserver.cc:533] Webserver started at http://127.16.193.65:40827/ using document root <none> and password file <none>
I20260709 12:15:43.471511 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.471666 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.471876 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.473115 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "a5f1ac44ab684972a0d851b94e427187"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.477330 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:43.480379 17752 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.481055 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:15:43.481287 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "a5f1ac44ab684972a0d851b94e427187"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.481518 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.494313 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.495491 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.496814 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:43.498962 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:43.499151 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.499452 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:43.499619 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.537209 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:44899
I20260709 12:15:43.537297 17814 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:44899 every 8 connection(s)
I20260709 12:15:43.542128 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.548465 17819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.549916 17820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.554898 17815 heartbeater.cc:344] Connected to a master server at 127.16.193.126:44857
I20260709 12:15:43.555459 17815 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:43.556757 17815 heartbeater.cc:507] Master 127.16.193.126:44857 requested a full tablet report, sending...
I20260709 12:15:43.558045 17157 server_base.cc:1061] running on GCE node
W20260709 12:15:43.558525 17822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.559597 17688 ts_manager.cc:194] Registered new tserver with Master: a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899)
I20260709 12:15:43.560096 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.560391 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.560572 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599343560542 us; error 0 us; skew 500 ppm
I20260709 12:15:43.561115 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.561501 17688 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46066
I20260709 12:15:43.563702 17157 webserver.cc:533] Webserver started at http://127.16.193.66:36553/ using document root <none> and password file <none>
I20260709 12:15:43.564203 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.564404 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.564664 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.565721 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.569808 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:15:43.572911 17827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.573590 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:15:43.573897 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.574201 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.594460 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.595713 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.597116 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:43.599334 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:43.599534 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.599769 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:43.599961 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.636991 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:40887
I20260709 12:15:43.637125 17889 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:40887 every 8 connection(s)
I20260709 12:15:43.641435 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.648672 17893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.650803 17894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.652341 17157 server_base.cc:1061] running on GCE node
W20260709 12:15:43.652911 17896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.653908 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.654173 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.654193 17890 heartbeater.cc:344] Connected to a master server at 127.16.193.126:44857
I20260709 12:15:43.654542 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599343654524 us; error 0 us; skew 500 ppm
I20260709 12:15:43.654677 17890 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:43.655169 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.655516 17890 heartbeater.cc:507] Master 127.16.193.126:44857 requested a full tablet report, sending...
I20260709 12:15:43.657722 17688 ts_manager.cc:194] Registered new tserver with Master: a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:43.657806 17157 webserver.cc:533] Webserver started at http://127.16.193.67:38171/ using document root <none> and password file <none>
I20260709 12:15:43.658452 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.658690 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.658994 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.659525 17688 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54354
I20260709 12:15:43.660511 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "13fcc17f52a445e4b00ac27e881929c4"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.664688 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:43.667726 17901 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.668426 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:15:43.668687 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "13fcc17f52a445e4b00ac27e881929c4"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-n640"
I20260709 12:15:43.668948 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.698010 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.699223 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.700783 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:43.702939 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:43.703105 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.703296 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:43.703430 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.740582 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:43297
I20260709 12:15:43.740684 17963 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:43297 every 8 connection(s)
I20260709 12:15:43.753672 17964 heartbeater.cc:344] Connected to a master server at 127.16.193.126:44857
I20260709 12:15:43.754019 17964 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:43.754739 17964 heartbeater.cc:507] Master 127.16.193.126:44857 requested a full tablet report, sending...
I20260709 12:15:43.756517 17688 ts_manager.cc:194] Registered new tserver with Master: 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:43.756909 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01311198s
I20260709 12:15:43.757818 17688 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54358
I20260709 12:15:43.780107 17688 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54368:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:43.783342 17688 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:43.851689 17929 tablet_service.cc:1511] Processing CreateTablet for tablet ddbdd2bf1a034c03b48b38c4f27d11f6 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:15:43.853255 17929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbdd2bf1a034c03b48b38c4f27d11f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.853770 17928 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6ad8230c374b4287be43bf7cb1f75c (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:15:43.855094 17928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6ad8230c374b4287be43bf7cb1f75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.864269 17780 tablet_service.cc:1511] Processing CreateTablet for tablet ddbdd2bf1a034c03b48b38c4f27d11f6 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:15:43.865626 17780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbdd2bf1a034c03b48b38c4f27d11f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.866915 17927 tablet_service.cc:1511] Processing CreateTablet for tablet b0dbdad2aed64b53a19eebfd1a1fc34f (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:15:43.868216 17927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dbdad2aed64b53a19eebfd1a1fc34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.868286 17779 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6ad8230c374b4287be43bf7cb1f75c (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:15:43.869585 17779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6ad8230c374b4287be43bf7cb1f75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.872423 17778 tablet_service.cc:1511] Processing CreateTablet for tablet b0dbdad2aed64b53a19eebfd1a1fc34f (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:15:43.874765 17777 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7143273fee494e9754af0a4aafda1d (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:15:43.880316 17926 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7143273fee494e9754af0a4aafda1d (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:15:43.881697 17926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7143273fee494e9754af0a4aafda1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.881801 17776 tablet_service.cc:1511] Processing CreateTablet for tablet 14329fa0f0404c30b0da2fac0199f9a2 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:15:43.884105 17778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dbdad2aed64b53a19eebfd1a1fc34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.885900 17775 tablet_service.cc:1511] Processing CreateTablet for tablet 51a0fa871a3a49b7b9720abfff3ae9b9 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:15:43.887413 17774 tablet_service.cc:1511] Processing CreateTablet for tablet 0331d62092a3425f93c053ff120a900a (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:15:43.891346 17772 tablet_service.cc:1511] Processing CreateTablet for tablet 7561ca2cc72147d9ab31e3f38da55b7e (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:15:43.892503 17773 tablet_service.cc:1511] Processing CreateTablet for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:15:43.894618 17771 tablet_service.cc:1511] Processing CreateTablet for tablet 0a054446fbe64c51a2615a2aceb73305 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:15:43.896155 17854 tablet_service.cc:1511] Processing CreateTablet for tablet 1e6ad8230c374b4287be43bf7cb1f75c (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:15:43.894641 17855 tablet_service.cc:1511] Processing CreateTablet for tablet ddbdd2bf1a034c03b48b38c4f27d11f6 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:15:43.898298 17851 tablet_service.cc:1511] Processing CreateTablet for tablet 14329fa0f0404c30b0da2fac0199f9a2 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:15:43.898497 17852 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7143273fee494e9754af0a4aafda1d (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:15:43.898319 17854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e6ad8230c374b4287be43bf7cb1f75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.900238 17853 tablet_service.cc:1511] Processing CreateTablet for tablet b0dbdad2aed64b53a19eebfd1a1fc34f (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:15:43.900604 17851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14329fa0f0404c30b0da2fac0199f9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.901206 17852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7143273fee494e9754af0a4aafda1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.901919 17855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbdd2bf1a034c03b48b38c4f27d11f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.909241 17928 tablet_service.cc:1511] Processing CreateTablet for tablet 51a0fa871a3a49b7b9720abfff3ae9b9 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:15:43.910499 17853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dbdad2aed64b53a19eebfd1a1fc34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.912186 17925 tablet_service.cc:1511] Processing CreateTablet for tablet 0331d62092a3425f93c053ff120a900a (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:15:43.914920 17924 tablet_service.cc:1511] Processing CreateTablet for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:15:43.915612 17773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.917352 17776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14329fa0f0404c30b0da2fac0199f9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.917479 17923 tablet_service.cc:1511] Processing CreateTablet for tablet 7561ca2cc72147d9ab31e3f38da55b7e (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:15:43.919579 17775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a0fa871a3a49b7b9720abfff3ae9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.920070 17922 tablet_service.cc:1511] Processing CreateTablet for tablet 0a054446fbe64c51a2615a2aceb73305 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:15:43.910542 17928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a0fa871a3a49b7b9720abfff3ae9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.922286 17774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0331d62092a3425f93c053ff120a900a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.927416 17923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7561ca2cc72147d9ab31e3f38da55b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.928222 17847 tablet_service.cc:1511] Processing CreateTablet for tablet 7561ca2cc72147d9ab31e3f38da55b7e (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:15:43.929107 17777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7143273fee494e9754af0a4aafda1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.930462 17848 tablet_service.cc:1511] Processing CreateTablet for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:15:43.929641 17924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.932431 17849 tablet_service.cc:1511] Processing CreateTablet for tablet 0331d62092a3425f93c053ff120a900a (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:15:43.934271 17846 tablet_service.cc:1511] Processing CreateTablet for tablet 0a054446fbe64c51a2615a2aceb73305 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:15:43.923930 17850 tablet_service.cc:1511] Processing CreateTablet for tablet 51a0fa871a3a49b7b9720abfff3ae9b9 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:15:43.935545 17771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a054446fbe64c51a2615a2aceb73305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.929634 17847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7561ca2cc72147d9ab31e3f38da55b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.938831 17848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c9c1c9e3b343a5b11d1cdb20d38ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.939565 17850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a0fa871a3a49b7b9720abfff3ae9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.940092 17772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7561ca2cc72147d9ab31e3f38da55b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.943542 17846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a054446fbe64c51a2615a2aceb73305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.947472 17925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0331d62092a3425f93c053ff120a900a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.947572 17849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0331d62092a3425f93c053ff120a900a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.955621 17922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a054446fbe64c51a2615a2aceb73305. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.967975 17979 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:43.968174 17929 tablet_service.cc:1511] Processing CreateTablet for tablet 14329fa0f0404c30b0da2fac0199f9a2 (DEFAULT_TABLE table=test-workload [id=63391dd91031452893df3648a806d00a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:15:43.969597 17929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14329fa0f0404c30b0da2fac0199f9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.009552 17979 tablet_bootstrap.cc:654] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.038192 17980 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.038792 17981 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.047024 17981 tablet_bootstrap.cc:654] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.048039 17980 tablet_bootstrap.cc:654] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.051857 17979 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.052445 17979 ts_tablet_manager.cc:1403] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.085s	user 0.005s	sys 0.012s
I20260709 12:15:44.056308 17979 raft_consensus.cc:359] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.057338 17981 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.057281 17979 raft_consensus.cc:385] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.057865 17981 ts_tablet_manager.cc:1403] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.002s
I20260709 12:15:44.057787 17979 raft_consensus.cc:740] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.058873 17979 consensus_queue.cc:260] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.060593 17981 raft_consensus.cc:359] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.061136 17981 raft_consensus.cc:385] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.061412 17981 raft_consensus.cc:740] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.062052 17980 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.062068 17981 consensus_queue.cc:260] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.062748 17980 ts_tablet_manager.cc:1403] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.010s
I20260709 12:15:44.064328 17890 heartbeater.cc:499] Master 127.16.193.126:44857 was elected leader, sending a full tablet report...
I20260709 12:15:44.064638 17981 ts_tablet_manager.cc:1434] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:15:44.065351 17981 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.065937 17980 raft_consensus.cc:359] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.066642 17980 raft_consensus.cc:385] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.066902 17980 raft_consensus.cc:740] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.067804 17980 consensus_queue.cc:260] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.071784 17980 ts_tablet_manager.cc:1434] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:15:44.071796 17979 ts_tablet_manager.cc:1434] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.019s	user 0.014s	sys 0.001s
I20260709 12:15:44.071974 17981 tablet_bootstrap.cc:654] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.073055 17964 heartbeater.cc:499] Master 127.16.193.126:44857 was elected leader, sending a full tablet report...
I20260709 12:15:44.073606 17980 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.074517 17979 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.080927 17980 tablet_bootstrap.cc:654] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.081046 17979 tablet_bootstrap.cc:654] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.073348 17815 heartbeater.cc:499] Master 127.16.193.126:44857 was elected leader, sending a full tablet report...
I20260709 12:15:44.100222 17981 tablet_bootstrap.cc:492] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.100900 17981 ts_tablet_manager.cc:1403] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.005s
I20260709 12:15:44.106321 17980 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.106837 17980 ts_tablet_manager.cc:1403] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.011s
I20260709 12:15:44.106834 17981 raft_consensus.cc:359] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.107698 17981 raft_consensus.cc:385] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.108014 17981 raft_consensus.cc:740] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.108736 17981 consensus_queue.cc:260] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.110006 17979 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.109936 17980 raft_consensus.cc:359] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.110575 17979 ts_tablet_manager.cc:1403] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.011s
I20260709 12:15:44.110795 17980 raft_consensus.cc:385] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.111047 17981 ts_tablet_manager.cc:1434] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:15:44.111146 17980 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.112021 17981 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.112030 17980 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.113610 17979 raft_consensus.cc:359] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.115303 17980 ts_tablet_manager.cc:1434] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:15:44.114289 17979 raft_consensus.cc:385] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.116089 17979 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.116226 17980 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.117035 17979 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.118770 17981 tablet_bootstrap.cc:654] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.119346 17979 ts_tablet_manager.cc:1434] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.006s
I20260709 12:15:44.120282 17979 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.122788 17980 tablet_bootstrap.cc:654] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.128063 17979 tablet_bootstrap.cc:654] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.130429 17981 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.130897 17981 ts_tablet_manager.cc:1403] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 12:15:44.140564 17979 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.141045 17979 ts_tablet_manager.cc:1403] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:15:44.140940 17981 raft_consensus.cc:359] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.141681 17981 raft_consensus.cc:385] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.141976 17981 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.143440 17981 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.143880 17979 raft_consensus.cc:359] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.144536 17979 raft_consensus.cc:385] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.144836 17979 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.145890 17980 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.145890 17981 ts_tablet_manager.cc:1434] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:15:44.145761 17979 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.146535 17980 ts_tablet_manager.cc:1403] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.013s
I20260709 12:15:44.147007 17981 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.148137 17979 ts_tablet_manager.cc:1434] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:15:44.149041 17979 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.149349 17980 raft_consensus.cc:359] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.150098 17980 raft_consensus.cc:385] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.150410 17980 raft_consensus.cc:740] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.151283 17980 consensus_queue.cc:260] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.153378 17980 ts_tablet_manager.cc:1434] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:15:44.153980 17981 tablet_bootstrap.cc:654] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.154359 17980 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.161085 17980 tablet_bootstrap.cc:654] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.164924 17981 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.165407 17981 ts_tablet_manager.cc:1403] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:15:44.168248 17981 raft_consensus.cc:359] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.168978 17981 raft_consensus.cc:385] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.169262 17981 raft_consensus.cc:740] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.169984 17981 consensus_queue.cc:260] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.171021 17979 tablet_bootstrap.cc:654] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.172185 17981 ts_tablet_manager.cc:1434] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:15:44.173496 17981 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.176944 17980 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.177461 17980 ts_tablet_manager.cc:1403] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.009s
I20260709 12:15:44.178999 17985 raft_consensus.cc:493] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.179468 17985 raft_consensus.cc:515] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.180395 17981 tablet_bootstrap.cc:654] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.180716 17980 raft_consensus.cc:359] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.181407 17980 raft_consensus.cc:385] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.181720 17980 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.182436 17980 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.186167 17980 ts_tablet_manager.cc:1434] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:15:44.187098 17980 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.193158 17981 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.193321 17985 leader_election.cc:290] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.193531 17980 tablet_bootstrap.cc:654] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.193648 17981 ts_tablet_manager.cc:1403] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 12:15:44.196141 17979 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.196579 17979 ts_tablet_manager.cc:1403] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.003s
I20260709 12:15:44.196867 17981 raft_consensus.cc:359] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.197575 17981 raft_consensus.cc:385] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.197868 17981 raft_consensus.cc:740] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.198232 17986 raft_consensus.cc:493] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.198680 17986 raft_consensus.cc:515] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.198823 17981 consensus_queue.cc:260] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.206089 17986 leader_election.cc:290] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297), a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899)
I20260709 12:15:44.209596 17981 ts_tablet_manager.cc:1434] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 12:15:44.210834 17981 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.218580 17981 tablet_bootstrap.cc:654] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.223868 17987 raft_consensus.cc:493] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.224519 17987 raft_consensus.cc:515] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.228133 17987 leader_election.cc:290] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.245436 17979 raft_consensus.cc:359] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.246477 17979 raft_consensus.cc:385] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.246870 17979 raft_consensus.cc:740] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.254629 17865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
W20260709 12:15:44.259894 17903 leader_election.cc:343] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.263949 18000 raft_consensus.cc:493] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.264611 18000 raft_consensus.cc:515] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.267403 17986 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.267535 18000 leader_election.cc:290] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.268093 17986 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.270426 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.270722 17986 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.271361 17790 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.278354 17981 tablet_bootstrap.cc:492] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.278903 17981 ts_tablet_manager.cc:1403] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.068s	user 0.029s	sys 0.028s
I20260709 12:15:44.273626 17985 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.279524 17980 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.280014 17980 ts_tablet_manager.cc:1403] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.093s	user 0.040s	sys 0.050s
I20260709 12:15:44.279882 17985 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.281318 17981 raft_consensus.cc:359] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.281840 17981 raft_consensus.cc:385] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.281909 17985 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.282238 17981 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.282832 17980 raft_consensus.cc:359] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.283082 17981 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.283571 17980 raft_consensus.cc:385] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.283946 17980 raft_consensus.cc:740] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.284272 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.284722 17789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.285342 17789 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.286242 17939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.284689 17980 consensus_queue.cc:260] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.286639 17904 leader_election.cc:304] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.288125 17789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.291813 17985 raft_consensus.cc:2804] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.292169 17985 raft_consensus.cc:493] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.292497 17985 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.287032 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.293174 17790 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.286880 17939 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 0.
I20260709 12:15:44.288610 17980 ts_tablet_manager.cc:1434] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:15:44.288887 17865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.288157 17981 ts_tablet_manager.cc:1434] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:15:44.297343 17981 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.290017 17979 consensus_queue.cc:260] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.295688 17980 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.300093 17904 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.301326 17985 raft_consensus.cc:515] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.303508 17981 tablet_bootstrap.cc:654] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.305577 17985 leader_election.cc:290] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.306237 17985 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.309274 17979 ts_tablet_manager.cc:1434] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.112s	user 0.004s	sys 0.003s
I20260709 12:15:44.310369 17979 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.312505 17939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.312674 17937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.313113 17939 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.314126 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
W20260709 12:15:44.314359 17829 leader_election.cc:343] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.314805 17938 raft_consensus.cc:2468] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.317721 17979 tablet_bootstrap.cc:654] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.320299 17980 tablet_bootstrap.cc:654] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.306607 17985 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.321205 17985 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.288901 17830 leader_election.cc:304] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.324010 17986 raft_consensus.cc:2804] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.324393 17986 raft_consensus.cc:493] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.324731 17986 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.326334 17829 leader_election.cc:304] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:44.329401 18000 raft_consensus.cc:2804] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.330613 18000 raft_consensus.cc:493] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.331050 18000 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.337857 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.338586 17790 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:44.337940 17830 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:44899, user_credentials={real_user=slave}} blocked reactor thread for 49231.7us
I20260709 12:15:44.337857 17754 leader_election.cc:304] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.342765 17987 raft_consensus.cc:2804] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.343114 17987 raft_consensus.cc:493] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.343497 17987 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.344318 17979 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.344919 17979 ts_tablet_manager.cc:1403] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.006s
I20260709 12:15:44.345292 18000 raft_consensus.cc:515] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.349032 18000 leader_election.cc:290] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.349496 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.350242 17938 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.350282 17985 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.352593 17985 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.352998 17789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.353408 17789 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.348094 17865 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
W20260709 12:15:44.341303 17830 leader_election.cc:343] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.358940 17981 tablet_bootstrap.cc:492] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
W20260709 12:15:44.359161 17830 leader_election.cc:343] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.359506 17981 ts_tablet_manager.cc:1403] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.011s
I20260709 12:15:44.359112 17938 raft_consensus.cc:2468] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 1.
I20260709 12:15:44.359805 17830 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5; no voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.360652 18000 raft_consensus.cc:2749] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.360719 17829 leader_election.cc:304] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
W20260709 12:15:44.361156 17903 leader_election.cc:343] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.361655 18000 raft_consensus.cc:2804] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.362320 18000 raft_consensus.cc:697] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:44.363070 17788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.363191 18000 consensus_queue.cc:237] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.364667 17981 raft_consensus.cc:359] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.365186 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.359122 17789 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.365929 17864 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:44.365392 17830 leader_election.cc:343] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.367532 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.366863 17904 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.365404 17981 raft_consensus.cc:385] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.368799 17985 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.368938 17981 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.369616 17986 raft_consensus.cc:515] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.370121 17985 raft_consensus.cc:697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:44.370674 17979 raft_consensus.cc:359] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.371404 17979 raft_consensus.cc:385] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.371292 17985 consensus_queue.cc:237] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.364993 17790 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.371721 17979 raft_consensus.cc:740] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.372735 17904 leader_election.cc:304] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.373144 17979 consensus_queue.cc:260] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.373798 18009 raft_consensus.cc:2804] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.373553 17987 raft_consensus.cc:515] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.369745 17981 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.376384 18009 raft_consensus.cc:697] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:44.377142 17979 ts_tablet_manager.cc:1434] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 12:15:44.379274 17865 raft_consensus.cc:2393] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5f1ac44ab684972a0d851b94e427187 in current term 1: Already voted for candidate a2310c3a982443c7b2bdd928bcecb9e5 in this term.
I20260709 12:15:44.378638 17863 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.379966 17979 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.385061 17987 leader_election.cc:290] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 election: Requested vote from peers 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.385576 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.386160 17938 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.375548 17864 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.389446 17862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.390512 17862 raft_consensus.cc:2393] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a5f1ac44ab684972a0d851b94e427187 in current term 1: Already voted for candidate a2310c3a982443c7b2bdd928bcecb9e5 in this term.
I20260709 12:15:44.390641 17979 tablet_bootstrap.cc:654] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:44.390602 17736 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:44.377637 18009 consensus_queue.cc:237] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.404366 17939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.404532 17986 leader_election.cc:290] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Requested vote from peers 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297), a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899)
I20260709 12:15:44.405880 17986 raft_consensus.cc:493] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.406255 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.406423 17986 raft_consensus.cc:515] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.407321 17790 raft_consensus.cc:2393] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2310c3a982443c7b2bdd928bcecb9e5 in current term 1: Already voted for candidate a5f1ac44ab684972a0d851b94e427187 in this term.
I20260709 12:15:44.410267 17863 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.401132 17938 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 1.
I20260709 12:15:44.416905 17830 leader_election.cc:304] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5; no voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.417588 17981 ts_tablet_manager.cc:1434] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.058s	user 0.023s	sys 0.020s
I20260709 12:15:44.417446 17754 leader_election.cc:304] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.418682 17987 raft_consensus.cc:2804] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.420905 17980 tablet_bootstrap.cc:492] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.421347 17980 ts_tablet_manager.cc:1403] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.126s	user 0.020s	sys 0.020s
I20260709 12:15:44.420047 17981 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.424527 17979 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.424165 17980 raft_consensus.cc:359] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.418478 18000 raft_consensus.cc:2749] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.425788 17987 raft_consensus.cc:697] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:44.426404 17980 raft_consensus.cc:385] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.425869 17688 catalog_manager.cc:5697] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.428051 17979 ts_tablet_manager.cc:1403] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.048s	user 0.024s	sys 0.007s
I20260709 12:15:44.428609 17987 consensus_queue.cc:237] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:15:44.427994 17980 raft_consensus.cc:740] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.430887 17981 tablet_bootstrap.cc:654] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.431324 17979 raft_consensus.cc:359] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.432037 17979 raft_consensus.cc:385] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.431319 17980 consensus_queue.cc:260] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.432425 17979 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.432360 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.433209 17790 raft_consensus.cc:2468] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.441850 17986 leader_election.cc:290] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.449851 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.450009 17830 leader_election.cc:304] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.456103 17986 raft_consensus.cc:2804] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.456447 17986 raft_consensus.cc:493] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.456621 18019 raft_consensus.cc:493] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.456808 17986 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.457063 18019 raft_consensus.cc:515] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
W20260709 12:15:44.459550 17829 leader_election.cc:343] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.460158 18019 leader_election.cc:290] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.461570 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.461707 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.462375 17938 raft_consensus.cc:2393] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5f1ac44ab684972a0d851b94e427187 in current term 1: Already voted for candidate a2310c3a982443c7b2bdd928bcecb9e5 in this term.
I20260709 12:15:44.462826 17980 ts_tablet_manager.cc:1434] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.002s
I20260709 12:15:44.463603 17754 leader_election.cc:304] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5f1ac44ab684972a0d851b94e427187; no voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.441177 17979 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.469079 17979 ts_tablet_manager.cc:1434] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:15:44.469278 18019 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.472519 17980 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.472604 17979 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.470178 17685 catalog_manager.cc:5697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:44.480096 17979 tablet_bootstrap.cc:654] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.480770 17986 raft_consensus.cc:515] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.483803 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.484560 17790 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.486760 17985 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.487340 17985 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.489821 17985 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.492177 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.492599 17789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.492792 17863 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
W20260709 12:15:44.494043 17904 leader_election.cc:343] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.497223 17903 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.498904 17985 raft_consensus.cc:2804] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.499358 17985 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.499785 17985 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.503777 17980 tablet_bootstrap.cc:654] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.508184 17790 raft_consensus.cc:2468] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 1.
I20260709 12:15:44.509425 18019 raft_consensus.cc:2749] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.509773 17830 leader_election.cc:304] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.510812 18000 raft_consensus.cc:2804] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.511286 18000 raft_consensus.cc:697] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:44.512107 18000 consensus_queue.cc:237] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.512797 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.513284 17981 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.513746 17981 ts_tablet_manager.cc:1403] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.094s	user 0.032s	sys 0.012s
W20260709 12:15:44.514626 17829 leader_election.cc:343] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.516558 17981 raft_consensus.cc:359] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.517378 17981 raft_consensus.cc:385] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.517820 17981 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.517285 17985 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.519450 17986 leader_election.cc:290] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.518637 17981 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.521461 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.522297 17981 ts_tablet_manager.cc:1434] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:44.523394 17981 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.523618 17985 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.525398 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.521275 17687 catalog_manager.cc:5697] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.526079 17863 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.531322 17981 tablet_bootstrap.cc:654] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.531708 18013 raft_consensus.cc:493] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.532235 18013 raft_consensus.cc:515] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
W20260709 12:15:44.534574 17904 leader_election.cc:343] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.539199 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.539736 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.539911 17979 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.539913 17938 raft_consensus.cc:2468] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.540660 17979 ts_tablet_manager.cc:1403] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.068s	user 0.024s	sys 0.020s
W20260709 12:15:44.541038 17830 leader_election.cc:343] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.540336 18013 leader_election.cc:290] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.541605 17829 leader_election.cc:304] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.543030 18013 raft_consensus.cc:2804] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.544068 17979 raft_consensus.cc:359] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.545255 17863 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.546346 18013 raft_consensus.cc:493] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.547561 17979 raft_consensus.cc:385] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.547686 17903 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.550071 17979 raft_consensus.cc:740] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.551405 17985 raft_consensus.cc:2804] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.551308 17979 consensus_queue.cc:260] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.552181 17985 raft_consensus.cc:697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:44.553299 17985 consensus_queue.cc:237] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.554780 17979 ts_tablet_manager.cc:1434] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:15:44.556020 17979 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.546866 18013 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.574296 17980 tablet_bootstrap.cc:492] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.574888 17980 ts_tablet_manager.cc:1403] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.017s
I20260709 12:15:44.583050 17980 raft_consensus.cc:359] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.583966 17980 raft_consensus.cc:385] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.584441 17980 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.584093 18013 raft_consensus.cc:515] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.585372 17980 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.592010 17979 tablet_bootstrap.cc:654] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.587792 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.595450 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.586596 17685 catalog_manager.cc:5697] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:44.596194 17938 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:44.597993 17830 leader_election.cc:343] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.604928 17938 raft_consensus.cc:2468] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 1.
I20260709 12:15:44.606123 17829 leader_election.cc:304] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:44.606927 18000 raft_consensus.cc:2804] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.607362 18000 raft_consensus.cc:697] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:44.599117 17688 catalog_manager.cc:5697] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.609011 18013 leader_election.cc:290] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.620877 18000 consensus_queue.cc:237] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.626950 17979 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.627203 17981 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.627708 17981 ts_tablet_manager.cc:1403] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.105s	user 0.025s	sys 0.026s
I20260709 12:15:44.627704 17979 ts_tablet_manager.cc:1403] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.006s
I20260709 12:15:44.630373 17979 raft_consensus.cc:359] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.630865 17979 raft_consensus.cc:385] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.631094 17979 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.631760 17979 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.632546 17981 raft_consensus.cc:359] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.633214 17981 raft_consensus.cc:385] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.633508 17981 raft_consensus.cc:740] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.633896 17979 ts_tablet_manager.cc:1434] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:15:44.634326 17981 consensus_queue.cc:260] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.635591 18019 raft_consensus.cc:493] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.636271 17981 ts_tablet_manager.cc:1434] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:15:44.636224 18019 raft_consensus.cc:515] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.636869 17980 ts_tablet_manager.cc:1434] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 12:15:44.637161 17981 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: Bootstrap starting.
I20260709 12:15:44.637815 17980 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.638298 18019 leader_election.cc:290] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.639957 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.634725 17979 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
W20260709 12:15:44.643822 17754 leader_election.cc:343] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.649624 17981 tablet_bootstrap.cc:654] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.646374 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.664338 17979 tablet_bootstrap.cc:654] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:44.664575 17754 leader_election.cc:343] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:44.663993 17687 catalog_manager.cc:5697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:44.665023 17754 leader_election.cc:304] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5f1ac44ab684972a0d851b94e427187; no voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.666244 18019 raft_consensus.cc:2749] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.673400 17980 tablet_bootstrap.cc:654] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.674515 17979 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.675021 17979 ts_tablet_manager.cc:1403] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260709 12:15:44.677300 17979 raft_consensus.cc:359] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.677225 17688 catalog_manager.cc:5697] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.678789 17981 tablet_bootstrap.cc:492] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: No bootstrap required, opened a new log
I20260709 12:15:44.677966 17979 raft_consensus.cc:385] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.679206 17981 ts_tablet_manager.cc:1403] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent bootstrapping tablet: real 0.042s	user 0.023s	sys 0.005s
I20260709 12:15:44.679395 17979 raft_consensus.cc:740] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.680437 17980 tablet_bootstrap.cc:492] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.680116 17979 consensus_queue.cc:260] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.680821 17980 ts_tablet_manager.cc:1403] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.000s
I20260709 12:15:44.681881 17981 raft_consensus.cc:359] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.682281 17979 ts_tablet_manager.cc:1434] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:15:44.682544 17981 raft_consensus.cc:385] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.682816 17981 raft_consensus.cc:740] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.683202 17979 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: Bootstrap starting.
I20260709 12:15:44.683439 17981 consensus_queue.cc:260] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.683518 17980 raft_consensus.cc:359] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.684190 17980 raft_consensus.cc:385] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.684835 17980 raft_consensus.cc:740] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.685362 17981 ts_tablet_manager.cc:1434] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:15:44.685526 17980 consensus_queue.cc:260] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.687525 17980 ts_tablet_manager.cc:1434] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:15:44.688377 17980 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.688961 17979 tablet_bootstrap.cc:654] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.700632 17980 tablet_bootstrap.cc:654] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.703382 17979 tablet_bootstrap.cc:492] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: No bootstrap required, opened a new log
I20260709 12:15:44.703823 17979 ts_tablet_manager.cc:1403] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.015s
I20260709 12:15:44.705456 17980 tablet_bootstrap.cc:492] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.705857 17980 ts_tablet_manager.cc:1403] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 12:15:44.706208 17979 raft_consensus.cc:359] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.706902 17979 raft_consensus.cc:385] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.707158 17979 raft_consensus.cc:740] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.707777 17979 consensus_queue.cc:260] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.708364 17980 raft_consensus.cc:359] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.709427 17980 raft_consensus.cc:385] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.709712 17980 raft_consensus.cc:740] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.709828 17979 ts_tablet_manager.cc:1434] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:15:44.710342 17980 consensus_queue.cc:260] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.712390 17980 ts_tablet_manager.cc:1434] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:15:44.713209 17980 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: Bootstrap starting.
I20260709 12:15:44.717947 17980 tablet_bootstrap.cc:654] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.722636 17980 tablet_bootstrap.cc:492] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: No bootstrap required, opened a new log
I20260709 12:15:44.723117 17980 ts_tablet_manager.cc:1403] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: Time spent bootstrapping tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:15:44.724138 17985 raft_consensus.cc:493] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.724524 17985 raft_consensus.cc:515] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.725081 17980 raft_consensus.cc:359] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.725600 17980 raft_consensus.cc:385] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.725813 17980 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.726225 17985 leader_election.cc:290] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.726473 17980 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.726913 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.727496 17790 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.727924 17980 ts_tablet_manager.cc:1434] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:15:44.728332 17904 leader_election.cc:304] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.728997 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.729077 17985 raft_consensus.cc:2804] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.729444 17863 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.729458 17985 raft_consensus.cc:493] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.729918 17985 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.734643 18019 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.734740 17985 raft_consensus.cc:515] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.735049 18019 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.736513 17985 leader_election.cc:290] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.736773 18019 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.737437 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.737627 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.737927 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.738001 17790 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.738121 17863 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.738479 17864 raft_consensus.cc:2393] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5f1ac44ab684972a0d851b94e427187 in current term 1: Already voted for candidate 13fcc17f52a445e4b00ac27e881929c4 in this term.
I20260709 12:15:44.739825 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.741109 17754 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5f1ac44ab684972a0d851b94e427187; no voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.742188 18019 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.744782 17863 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.744882 17790 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.745911 17903 leader_election.cc:304] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:44.746074 18000 raft_consensus.cc:493] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.746547 18000 raft_consensus.cc:515] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.746830 17985 raft_consensus.cc:2804] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.747363 17985 raft_consensus.cc:697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:44.748154 17985 consensus_queue.cc:237] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.749269 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7561ca2cc72147d9ab31e3f38da55b7e" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.749907 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7561ca2cc72147d9ab31e3f38da55b7e" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.749986 17790 raft_consensus.cc:2468] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.750563 17938 raft_consensus.cc:2468] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 0.
I20260709 12:15:44.751258 17830 leader_election.cc:304] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.749354 18019 raft_consensus.cc:2749] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.751991 18013 raft_consensus.cc:2804] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.752362 18013 raft_consensus.cc:493] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.752741 18013 raft_consensus.cc:3060] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.748549 18000 leader_election.cc:290] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.758878 18013 raft_consensus.cc:515] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.760474 18013 leader_election.cc:290] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.760056 17687 catalog_manager.cc:5697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:44.761247 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7561ca2cc72147d9ab31e3f38da55b7e" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.761439 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7561ca2cc72147d9ab31e3f38da55b7e" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:44.761818 17938 raft_consensus.cc:3060] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.761766 17790 raft_consensus.cc:3060] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.768416 17938 raft_consensus.cc:2468] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 1.
I20260709 12:15:44.768416 17790 raft_consensus.cc:2468] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 1.
I20260709 12:15:44.769428 17829 leader_election.cc:304] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:44.770170 18013 raft_consensus.cc:2804] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.770632 18013 raft_consensus.cc:697] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:44.771723 18013 consensus_queue.cc:237] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.778172 17687 catalog_manager.cc:5697] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.781786 17985 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.793170 18014 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:44.805800 17985 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.817916 18014 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.843456 18019 raft_consensus.cc:493] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.843940 18019 raft_consensus.cc:515] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.844888 18025 consensus_queue.cc:1048] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.845707 18019 leader_election.cc:290] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:44.846395 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.846699 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fcc17f52a445e4b00ac27e881929c4" is_pre_election: true
I20260709 12:15:44.847443 17938 raft_consensus.cc:2393] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5f1ac44ab684972a0d851b94e427187 in current term 1: Already voted for candidate a2310c3a982443c7b2bdd928bcecb9e5 in this term.
I20260709 12:15:44.848349 17754 leader_election.cc:304] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5f1ac44ab684972a0d851b94e427187; no voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:44.848513 18014 raft_consensus.cc:493] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.849141 18019 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.849066 18014 raft_consensus.cc:515] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.852644 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" is_pre_election: true
I20260709 12:15:44.852454 18014 leader_election.cc:290] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.853385 17864 raft_consensus.cc:2468] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.853724 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187" is_pre_election: true
I20260709 12:15:44.854403 17903 leader_election.cc:304] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:44.854651 17790 raft_consensus.cc:2468] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 0.
I20260709 12:15:44.856318 18014 raft_consensus.cc:2804] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:44.856237 18022 consensus_queue.cc:1048] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:44.856673 18014 raft_consensus.cc:493] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.857046 18014 raft_consensus.cc:3060] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.857127 18019 raft_consensus.cc:2749] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:44.865165 18014 raft_consensus.cc:515] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.868185 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:44.868736 17790 raft_consensus.cc:3060] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.871039 18014 leader_election.cc:290] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:44.871573 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a054446fbe64c51a2615a2aceb73305" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:44.872149 17863 raft_consensus.cc:3060] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.874413 17790 raft_consensus.cc:2468] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.875512 17904 leader_election.cc:304] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:44.876274 18014 raft_consensus.cc:2804] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:44.876739 18014 raft_consensus.cc:697] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:44.877512 18014 consensus_queue.cc:237] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:44.878554 17863 raft_consensus.cc:2468] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 1.
I20260709 12:15:44.884308 17688 catalog_manager.cc:5697] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:44.919579 17938 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:44.925594 18013 consensus_queue.cc:1048] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.936898 17986 consensus_queue.cc:1048] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.983991 17986 consensus_queue.cc:1048] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:44.993569 18013 consensus_queue.cc:1048] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.039747 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:45.040293 17938 raft_consensus.cc:606] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:45.045055 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:45.045539 17938 raft_consensus.cc:606] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:45.053781 18014 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.059610 17863 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:44064
I20260709 12:15:45.060180 17863 raft_consensus.cc:606] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:45.064157 18014 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.069934 17864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:44064
I20260709 12:15:45.070497 17864 raft_consensus.cc:606] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:45.076247 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:45.076691 17938 raft_consensus.cc:606] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Received request to transfer leadership to TS a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:45.084285 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:45.084685 17938 raft_consensus.cc:606] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Received request to transfer leadership to TS a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:15:45.160140 18013 consensus_queue.cc:1048] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.170022 18013 consensus_queue.cc:1048] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.181116 18013 consensus_queue.cc:1048] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.192427 17986 consensus_queue.cc:1048] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.201787 17986 raft_consensus.cc:993] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.202301 18000 raft_consensus.cc:1081] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.203810 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46150
I20260709 12:15:45.204416 17790 raft_consensus.cc:493] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.204857 17790 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.209604 17790 raft_consensus.cc:515] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.211320 17790 leader_election.cc:290] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.212020 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:45.212263 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7143273fee494e9754af0a4aafda1d" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.212634 17863 raft_consensus.cc:3055] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.212803 17938 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.212983 17863 raft_consensus.cc:740] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:45.213864 17863 consensus_queue.cc:260] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.215058 17863 raft_consensus.cc:3060] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.215337 17985 raft_consensus.cc:993] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.215770 18009 raft_consensus.cc:1081] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.217100 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46140
I20260709 12:15:45.217684 17790 raft_consensus.cc:493] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.217993 17790 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.219527 17938 raft_consensus.cc:2468] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.220551 17754 leader_election.cc:304] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.221303 18019 raft_consensus.cc:2804] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.221635 17863 raft_consensus.cc:2468] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.221747 18019 raft_consensus.cc:697] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:45.222640 18019 consensus_queue.cc:237] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.223786 17790 raft_consensus.cc:515] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.225741 17790 leader_election.cc:290] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.225616 18009 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.227337 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.227792 17938 raft_consensus.cc:3055] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.228111 17938 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:45.228150 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:45.228770 17863 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.229493 17938 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.230861 17938 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.231108 17687 catalog_manager.cc:5697] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 1 to 2, leader changed from a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.235284 17863 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.236430 17754 leader_election.cc:304] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.237193 18019 raft_consensus.cc:2804] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.237648 18019 raft_consensus.cc:697] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:45.238401 18019 consensus_queue.cc:237] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.244128 18008 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.245800 17938 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.247654 17685 catalog_manager.cc:5697] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 1 to 2, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.251132 18008 raft_consensus.cc:993] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:15:45.251583 17985 raft_consensus.cc:1081] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Signalling peer a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:15:45.252883 17864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:44054
I20260709 12:15:45.253330 17864 raft_consensus.cc:493] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.253701 17864 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.259878 17864 raft_consensus.cc:515] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.262488 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:45.263046 17790 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.263317 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.263860 17938 raft_consensus.cc:3055] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.264192 17938 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:45.264961 17938 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.265995 17938 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.267335 17864 leader_election.cc:290] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.269623 17790 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:15:45.270620 17830 leader_election.cc:304] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.271369 18000 raft_consensus.cc:2804] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.271848 18000 raft_consensus.cc:697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:45.272517 17938 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:15:45.272640 18000 consensus_queue.cc:237] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.278879 17685 catalog_manager.cc:5697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 1 to 2, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.318005 17986 raft_consensus.cc:993] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.318455 18000 raft_consensus.cc:1081] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.319893 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46150
I20260709 12:15:45.320375 17790 raft_consensus.cc:493] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.320691 17790 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.325115 17790 raft_consensus.cc:515] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.326539 17790 leader_election.cc:290] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.327364 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:45.327684 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14329fa0f0404c30b0da2fac0199f9a2" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.327960 17864 raft_consensus.cc:3055] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.328224 17938 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.328303 17864 raft_consensus.cc:740] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:45.329149 17864 consensus_queue.cc:260] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.330323 17864 raft_consensus.cc:3060] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.335042 17938 raft_consensus.cc:2468] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.336540 17754 leader_election.cc:304] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.336975 18008 raft_consensus.cc:993] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.336999 17864 raft_consensus.cc:2468] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.337347 18019 raft_consensus.cc:2804] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.337508 17985 raft_consensus.cc:1081] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:45.337823 18019 raft_consensus.cc:697] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:45.338629 18019 consensus_queue.cc:237] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.339073 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46140
I20260709 12:15:45.339685 17790 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.340211 17790 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.341944 17985 consensus_queue.cc:1048] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.346398 17790 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.346257 17687 catalog_manager.cc:5697] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 1 to 2, leader changed from a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.349129 17863 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:45.349711 17863 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.356763 18009 consensus_queue.cc:1048] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.358331 17863 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.365549 17790 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.365532 17754 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.366436 18019 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.366899 18019 raft_consensus.cc:697] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:45.367842 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.367718 18019 consensus_queue.cc:237] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.368453 17938 raft_consensus.cc:3055] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.368757 17938 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:45.369509 17938 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.370831 17938 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.375478 17685 catalog_manager.cc:5697] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 1 to 2, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.376120 17938 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:45.534174 18009 raft_consensus.cc:993] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:15:45.534663 18008 raft_consensus.cc:1081] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Signalling peer a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:15:45.536115 17863 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:44054
I20260709 12:15:45.536687 17863 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.537082 17863 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.542764 17863 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.544291 17863 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:45.544948 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:45.545207 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:45.545559 17790 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.545708 17938 raft_consensus.cc:3055] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:45.546093 17938 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:45.546695 17938 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.547578 17938 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:45.550042 17790 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:15:45.550928 17830 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:45.551576 18000 raft_consensus.cc:2804] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:45.551961 18000 raft_consensus.cc:697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:45.552044 17938 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:15:45.552781 18000 consensus_queue.cc:237] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:45.558590 17685 catalog_manager.cc:5697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 1 to 2, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.691135 17863 raft_consensus.cc:1275] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.692489 18019 consensus_queue.cc:1048] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.692714 17938 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.694109 18019 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.704246 17864 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.708704 18071 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.705814 17938 raft_consensus.cc:1275] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.716933 18072 consensus_queue.cc:1048] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.736593 17938 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.737926 18000 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.738595 17863 raft_consensus.cc:1275] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.739929 18072 consensus_queue.cc:1048] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.747134 17790 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.748937 18073 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.752574 17939 raft_consensus.cc:1275] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.754112 18071 consensus_queue.cc:1048] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.787817 17939 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.788856 18071 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.796955 17863 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.798784 18019 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.983713 17939 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.984841 18000 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:45.991712 17790 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:45.993139 18073 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:48.118484 17157 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:15:48.119151 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:48.134374 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:15:48.134809 17790 raft_consensus.cc:606] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Received request to transfer leadership to TS 13fcc17f52a445e4b00ac27e881929c4
I20260709 12:15:48.136821 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:15:48.137178 17790 raft_consensus.cc:606] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Received request to transfer leadership to TS 13fcc17f52a445e4b00ac27e881929c4
I20260709 12:15:48.138000 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:48.138422 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:48.407346 18078 raft_consensus.cc:993] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: : Instructing follower 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:15:48.407815 18072 raft_consensus.cc:1081] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Signalling peer 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:15:48.409148 17939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:49778
I20260709 12:15:48.409704 17939 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:48.410029 17939 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.415194 17939 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.417106 18078 raft_consensus.cc:993] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: : Instructing follower 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:15:48.417049 17939 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 3 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:48.417659 18078 raft_consensus.cc:1081] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Signalling peer 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:15:48.417878 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:48.418118 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:48.418423 17790 raft_consensus.cc:3055] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:48.418737 17864 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.418768 17790 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:48.419435 17939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:49778
I20260709 12:15:48.419956 17939 raft_consensus.cc:493] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:48.419684 17790 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.420369 17939 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.420791 17790 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.424988 17864 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 3.
I20260709 12:15:48.426021 17903 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:48.426712 18008 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:48.426545 17939 raft_consensus.cc:515] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.427709 17790 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 3.
I20260709 12:15:48.428742 18008 raft_consensus.cc:697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:48.429039 17939 leader_election.cc:290] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 3 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:15:48.429632 18008 consensus_queue.cc:237] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.431144 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:15:48.431262 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:48.431677 17790 raft_consensus.cc:3055] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:48.431839 17864 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.432024 17790 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:48.432839 17790 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.434331 17790 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:48.438364 17864 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 3.
I20260709 12:15:48.439662 17903 leader_election.cc:304] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:15:48.440529 18079 raft_consensus.cc:2804] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:48.447908 18079 raft_consensus.cc:697] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:48.448045 17790 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 3.
I20260709 12:15:48.448850 18079 consensus_queue.cc:237] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:48.454321 17687 catalog_manager.cc:5697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 2 to 3, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:48.468420 17687 catalog_manager.cc:5697] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 2 to 3, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:48.854851 17864 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:48.856074 18079 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:48.864765 17790 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:48.870153 18008 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:48.917750 17790 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:48.919160 18008 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:48.926900 17864 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:48.928576 18008 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:49.139495 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:49.144572 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:49.144922 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:50.145712 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:50.150794 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:50.151149 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:51.152108 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:51.157629 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:51.158034 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:52.158838 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:52.164290 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:52.164705 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:53.165903 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:53.171955 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:53.172325 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:54.173394 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.179817 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:54.180193 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:55.181030 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.186506 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:55.186858 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:56.187638 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:56.192698 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:56.193063 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:57.193912 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:57.198956 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:57.199371 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:58.369989 17157 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:15:58.376349 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:58.376972 17938 raft_consensus.cc:606] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:58.381191 17938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:49794
I20260709 12:15:58.381600 17938 raft_consensus.cc:606] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:58.390053 17864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:44064
I20260709 12:15:58.390518 17864 raft_consensus.cc:606] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:58.394693 17864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0331d62092a3425f93c053ff120a900a"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:44064
I20260709 12:15:58.395156 17864 raft_consensus.cc:606] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
I20260709 12:15:58.399211 17864 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
mode: GRACEFUL
new_leader_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:44064
I20260709 12:15:58.399662 17864 raft_consensus.cc:606] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Received request to transfer leadership to TS a5f1ac44ab684972a0d851b94e427187
W20260709 12:15:58.400575 17960 debug-util.cc:398] Leaking SignalData structure 0x7b08000abd20 after lost signal to thread 17160
W20260709 12:15:58.401419 17960 debug-util.cc:398] Leaking SignalData structure 0x7b0800030280 after lost signal to thread 17722
W20260709 12:15:58.402410 17960 debug-util.cc:398] Leaking SignalData structure 0x7b08000967e0 after lost signal to thread 17814
W20260709 12:15:58.403272 17960 debug-util.cc:398] Leaking SignalData structure 0x7b08000c36a0 after lost signal to thread 17889
W20260709 12:15:58.404105 17960 debug-util.cc:398] Leaking SignalData structure 0x7b080012a660 after lost signal to thread 17963
I20260709 12:15:58.444674 18107 raft_consensus.cc:993] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.445263 18100 raft_consensus.cc:1081] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.446725 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46140
I20260709 12:15:58.447185 17790 raft_consensus.cc:493] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.447579 17790 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.452643 17790 raft_consensus.cc:515] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.454416 17790 leader_election.cc:290] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 4 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:58.455505 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:58.455385 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dbdad2aed64b53a19eebfd1a1fc34f" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:58.456180 17938 raft_consensus.cc:3055] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:58.456588 17938 raft_consensus.cc:740] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:58.457233 17938 consensus_queue.cc:260] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.458204 17938 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.456212 17864 raft_consensus.cc:3060] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.465286 17938 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 4.
I20260709 12:15:58.466585 17754 leader_election.cc:304] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:58.466615 17864 raft_consensus.cc:2468] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 4.
I20260709 12:15:58.467633 18105 raft_consensus.cc:2804] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:58.468608 18105 raft_consensus.cc:697] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:58.469797 18105 consensus_queue.cc:237] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.480437 17685 catalog_manager.cc:5697] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 3 to 4, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 4 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.504094 18114 raft_consensus.cc:993] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.504665 18095 raft_consensus.cc:1081] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.506043 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46150
I20260709 12:15:58.506583 17790 raft_consensus.cc:493] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.506927 17790 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.514195 17790 raft_consensus.cc:515] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.516116 17790 leader_election.cc:290] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 3 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:58.516722 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:58.517024 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:58.517251 17864 raft_consensus.cc:3055] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:58.517483 17938 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.517539 17864 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:58.518332 17864 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.519523 17864 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.523321 17938 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 3.
I20260709 12:15:58.524441 17754 leader_election.cc:304] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:58.525171 18105 raft_consensus.cc:2804] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:58.525552 17864 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 3.
I20260709 12:15:58.525671 18105 raft_consensus.cc:697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:58.526664 18105 consensus_queue.cc:237] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.535079 17685 catalog_manager.cc:5697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 2 to 3, leader changed from a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.645840 18107 raft_consensus.cc:993] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.646309 18100 raft_consensus.cc:1081] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.647861 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46140
I20260709 12:15:58.648293 17790 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.648562 17790 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.652613 17790 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.654007 17790 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 4 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:58.654738 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:58.655292 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:58.655313 17864 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.655875 17938 raft_consensus.cc:3055] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:58.656169 17938 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:15:58.656924 17938 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.658047 17938 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:58.661437 17864 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 4.
I20260709 12:15:58.662426 17754 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2310c3a982443c7b2bdd928bcecb9e5, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:58.663161 18105 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:58.663691 18105 raft_consensus.cc:697] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:58.664369 17938 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 4.
I20260709 12:15:58.664548 18105 consensus_queue.cc:237] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.672309 17685 catalog_manager.cc:5697] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 3 to 4, leader changed from 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 4 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.706827 18114 raft_consensus.cc:993] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.707235 18114 raft_consensus.cc:1081] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.708597 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0331d62092a3425f93c053ff120a900a"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46150
I20260709 12:15:58.708997 17790 raft_consensus.cc:493] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.709242 17790 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:58.713394 17790 raft_consensus.cc:515] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.714835 17790 leader_election.cc:290] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:58.715518 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:58.715842 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0331d62092a3425f93c053ff120a900a" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:58.716089 17864 raft_consensus.cc:3055] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:58.716414 17938 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:58.716418 17864 raft_consensus.cc:740] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:58.717388 17864 consensus_queue.cc:260] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.718649 17864 raft_consensus.cc:3060] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:58.720841 17938 raft_consensus.cc:2468] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:58.721750 17754 leader_election.cc:304] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:58.722390 18105 raft_consensus.cc:2804] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:58.722810 18105 raft_consensus.cc:697] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:58.723577 18105 consensus_queue.cc:237] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.726039 17864 raft_consensus.cc:2468] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 2.
I20260709 12:15:58.731714 17685 catalog_manager.cc:5697] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 1 to 2, leader changed from a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.817236 18095 raft_consensus.cc:993] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: : Instructing follower a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.817718 18114 raft_consensus.cc:1081] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Signalling peer a5f1ac44ab684972a0d851b94e427187 to start an election
I20260709 12:15:58.819173 17790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
 from {username='slave'} at 127.0.0.1:46150
I20260709 12:15:58.819772 17790 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.820124 17790 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.827039 17790 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.829020 17790 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 3 election: Requested vote from peers a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:15:58.830024 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:15:58.830024 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "a5f1ac44ab684972a0d851b94e427187" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:15:58.830756 17938 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.830861 17864 raft_consensus.cc:3055] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:58.831167 17864 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:15:58.831961 17864 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.833127 17864 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:58.837054 17938 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 3.
I20260709 12:15:58.838080 17754 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a5f1ac44ab684972a0d851b94e427187; no voters: 
I20260709 12:15:58.838851 18105 raft_consensus.cc:2804] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:58.838994 17864 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5f1ac44ab684972a0d851b94e427187 in term 3.
I20260709 12:15:58.839396 18105 raft_consensus.cc:697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Becoming Leader. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:15:58.840363 18105 consensus_queue.cc:237] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:15:58.848202 17685 catalog_manager.cc:5697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 reported cstate change: term changed from 2 to 3, leader changed from a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66) to a5f1ac44ab684972a0d851b94e427187 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "a5f1ac44ab684972a0d851b94e427187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.915989 17938 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:58.917042 18105 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:58.933771 17864 raft_consensus.cc:1275] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:58.935505 18108 consensus_queue.cc:1048] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:15:58.977828 17938 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:58.979333 18105 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:58.988479 17864 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:58.989763 18108 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.161358 17938 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:59.162485 18122 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:59.171139 17864 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:59.173353 18105 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:59.260535 17938 raft_consensus.cc:1275] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:59.261736 18105 consensus_queue.cc:1048] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:59.271785 17863 raft_consensus.cc:1275] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:59.273191 18122 consensus_queue.cc:1048] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:59.305559 17938 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.306753 18122 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:59.316502 17863 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Refusing update from remote peer a5f1ac44ab684972a0d851b94e427187: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:59.318284 18105 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:01.435391 17157 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:16:01.436100 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:01.443306 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:16:01.443773 17790 raft_consensus.cc:606] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Received request to transfer leadership to TS 13fcc17f52a445e4b00ac27e881929c4
I20260709 12:16:01.445892 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:16:01.446302 17790 raft_consensus.cc:606] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Received request to transfer leadership to TS 13fcc17f52a445e4b00ac27e881929c4
I20260709 12:16:01.448203 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:16:01.448596 17790 raft_consensus.cc:606] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Received request to transfer leadership to TS a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:16:01.450484 17790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6"
dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
mode: GRACEFUL
new_leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:46164
I20260709 12:16:01.450876 17790 raft_consensus.cc:606] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 LEADER]: Received request to transfer leadership to TS a2310c3a982443c7b2bdd928bcecb9e5
I20260709 12:16:01.451793 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:01.452208 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:01.585570 18133 raft_consensus.cc:993] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: : Instructing follower a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:16:01.586781 18105 raft_consensus.cc:1081] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Signalling peer a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:16:01.588304 17864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:16:01.588750 17864 raft_consensus.cc:493] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:01.589032 17864 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.593037 17864 raft_consensus.cc:515] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.594897 17864 leader_election.cc:290] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 4 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297)
I20260709 12:16:01.595582 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:16:01.596100 17790 raft_consensus.cc:3055] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:01.596019 17939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a0fa871a3a49b7b9720abfff3ae9b9" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:16:01.596444 17790 raft_consensus.cc:740] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:16:01.596668 17939 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.597190 17790 consensus_queue.cc:260] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.598069 17790 raft_consensus.cc:3060] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.601760 17939 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 4.
I20260709 12:16:01.602149 17790 raft_consensus.cc:2468] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 4.
I20260709 12:16:01.602842 17829 leader_election.cc:304] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:16:01.603711 18114 raft_consensus.cc:2804] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:01.604539 18114 raft_consensus.cc:697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:16:01.605163 18114 consensus_queue.cc:237] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.611419 17685 catalog_manager.cc:5697] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 3 to 4, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 4 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:01.875783 18105 raft_consensus.cc:993] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: : Instructing follower 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:16:01.876420 18133 raft_consensus.cc:1081] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Signalling peer 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:16:01.877863 17939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:49778
I20260709 12:16:01.878343 17939 raft_consensus.cc:493] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:01.878650 17939 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:01.882750 17939 raft_consensus.cc:515] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.884742 17939 leader_election.cc:290] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 5 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:16:01.885339 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:16:01.885723 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e6ad8230c374b4287be43bf7cb1f75c" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:16:01.885850 17790 raft_consensus.cc:3055] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:01.886225 17790 raft_consensus.cc:740] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:16:01.886233 17864 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:01.886895 17790 consensus_queue.cc:260] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.887777 17790 raft_consensus.cc:3060] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:01.891119 17864 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 5.
I20260709 12:16:01.892453 17903 leader_election.cc:304] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:16:01.892875 17790 raft_consensus.cc:2468] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 5.
I20260709 12:16:01.893143 18140 raft_consensus.cc:2804] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:01.894269 18140 raft_consensus.cc:697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 5 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:16:01.895097 18140 consensus_queue.cc:237] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.901244 17685 catalog_manager.cc:5697] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 4 to 5, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 5 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:01.932186 18105 raft_consensus.cc:993] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: : Instructing follower 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:16:01.932657 18133 raft_consensus.cc:1081] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Signalling peer 13fcc17f52a445e4b00ac27e881929c4 to start an election
I20260709 12:16:01.933889 17939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd"
dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
 from {username='slave'} at 127.0.0.1:49778
I20260709 12:16:01.934300 17939 raft_consensus.cc:493] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:01.934581 17939 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.938433 17939 raft_consensus.cc:515] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.939857 17939 leader_election.cc:290] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 4 election: Requested vote from peers a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899), a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887)
I20260709 12:16:01.940527 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:16:01.940796 17864 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c9c1c9e3b343a5b11d1cdb20d38ccd" candidate_uuid: "13fcc17f52a445e4b00ac27e881929c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
I20260709 12:16:01.940994 17790 raft_consensus.cc:3055] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:01.941299 17864 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.941304 17790 raft_consensus.cc:740] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:16:01.942067 17790 consensus_queue.cc:260] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.942903 17790 raft_consensus.cc:3060] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:01.945297 17864 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 4.
I20260709 12:16:01.946158 17903 leader_election.cc:304] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:16:01.946828 18140 raft_consensus.cc:2804] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:01.947049 17790 raft_consensus.cc:2468] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13fcc17f52a445e4b00ac27e881929c4 in term 4.
I20260709 12:16:01.947252 18140 raft_consensus.cc:697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 4 LEADER]: Becoming Leader. State: Replica: 13fcc17f52a445e4b00ac27e881929c4, State: Running, Role: LEADER
I20260709 12:16:01.948125 18140 consensus_queue.cc:237] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } }
I20260709 12:16:01.954351 17685 catalog_manager.cc:5697] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 reported cstate change: term changed from 3 to 4, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "13fcc17f52a445e4b00ac27e881929c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:01.986601 17790 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:01.988268 18114 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:01.999177 17939 raft_consensus.cc:1275] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:02.000507 18114 consensus_queue.cc:1048] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:02.035959 18105 raft_consensus.cc:993] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: : Instructing follower a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:16:02.036406 18133 raft_consensus.cc:1081] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 LEADER]: Signalling peer a2310c3a982443c7b2bdd928bcecb9e5 to start an election
I20260709 12:16:02.037654 17864 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6"
dest_uuid: "a2310c3a982443c7b2bdd928bcecb9e5"
 from {username='slave'} at 127.0.0.1:44056
I20260709 12:16:02.038056 17864 raft_consensus.cc:493] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:02.038336 17864 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:02.042173 17864 raft_consensus.cc:515] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:16:02.043529 17864 leader_election.cc:290] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Requested vote from peers 13fcc17f52a445e4b00ac27e881929c4 (127.16.193.67:43297), a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899)
I20260709 12:16:02.044292 17938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "13fcc17f52a445e4b00ac27e881929c4"
I20260709 12:16:02.044489 17790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbdd2bf1a034c03b48b38c4f27d11f6" candidate_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5f1ac44ab684972a0d851b94e427187"
I20260709 12:16:02.044822 17938 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:02.044958 17790 raft_consensus.cc:3055] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:02.045274 17790 raft_consensus.cc:740] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a5f1ac44ab684972a0d851b94e427187, State: Running, Role: LEADER
I20260709 12:16:02.046016 17790 consensus_queue.cc:260] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:16:02.047070 17790 raft_consensus.cc:3060] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:02.050650 17938 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:16:02.051604 17829 leader_election.cc:304] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13fcc17f52a445e4b00ac27e881929c4, a2310c3a982443c7b2bdd928bcecb9e5; no voters: 
I20260709 12:16:02.052273 18114 raft_consensus.cc:2804] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:02.052747 18114 raft_consensus.cc:697] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Becoming Leader. State: Replica: a2310c3a982443c7b2bdd928bcecb9e5, State: Running, Role: LEADER
I20260709 12:16:02.052803 17790 raft_consensus.cc:2468] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2310c3a982443c7b2bdd928bcecb9e5 in term 2.
I20260709 12:16:02.053642 18114 consensus_queue.cc:237] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } }
I20260709 12:16:02.060365 17687 catalog_manager.cc:5697] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 reported cstate change: term changed from 1 to 2, leader changed from a5f1ac44ab684972a0d851b94e427187 (127.16.193.65) to a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:02.393860 17864 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 5 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:02.395042 18140 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:02.395998 17864 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:02.397228 18140 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2310c3a982443c7b2bdd928bcecb9e5" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:02.408483 17790 raft_consensus.cc:1275] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 5 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:02.409711 18141 consensus_queue.cc:1048] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:02.418196 17789 raft_consensus.cc:1275] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Refusing update from remote peer 13fcc17f52a445e4b00ac27e881929c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:02.422864 18141 consensus_queue.cc:1048] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:02.453152 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:02.458642 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:02.459065 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:02.523427 17790 raft_consensus.cc:1275] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:02.524588 18114 consensus_queue.cc:1048] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f1ac44ab684972a0d851b94e427187" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:02.535444 17939 raft_consensus.cc:1275] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Refusing update from remote peer a2310c3a982443c7b2bdd928bcecb9e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:02.538434 18158 consensus_queue.cc:1048] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fcc17f52a445e4b00ac27e881929c4" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.460129 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:03.466084 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:03.466516 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:04.467489 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:04.472813 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:04.473281 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:05.474151 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:05.479867 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:05.480206 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:06.481151 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:06.489480 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:06.489907 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:07.490793 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:07.496079 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:07.496496 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:08.497552 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:08.503600 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:08.504027 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:09.504894 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:09.510300 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:09.510723 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:10.511652 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:10.516887 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:10.517313 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:11.545789 17157 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:16:11.546304 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
I20260709 12:16:11.575377 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:11.576140 17157 tablet_replica.cc:333] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.576781 17157 raft_consensus.cc:2243] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.577585 17157 raft_consensus.cc:2272] T 0f7143273fee494e9754af0a4aafda1d P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.579528 17157 tablet_replica.cc:333] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.580019 17157 raft_consensus.cc:2243] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.580410 17157 raft_consensus.cc:2272] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.582345 17157 tablet_replica.cc:333] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.582823 17157 raft_consensus.cc:2243] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:16:11.583017 17904 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 12:16:11.583757 17157 raft_consensus.cc:2272] T 0331d62092a3425f93c053ff120a900a P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.586024 17157 tablet_replica.cc:333] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
W20260709 12:16:11.586354 17904 consensus_peers.cc:597] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 -> Peer a5f1ac44ab684972a0d851b94e427187 (127.16.193.65:44899): Couldn't send request to peer a5f1ac44ab684972a0d851b94e427187. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:11.586553 17157 raft_consensus.cc:2243] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.587033 17157 raft_consensus.cc:2272] T 0a054446fbe64c51a2615a2aceb73305 P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.588819 17157 tablet_replica.cc:333] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.589454 17157 raft_consensus.cc:2243] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.589864 17157 raft_consensus.cc:2272] T 7561ca2cc72147d9ab31e3f38da55b7e P a5f1ac44ab684972a0d851b94e427187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.591532 17157 tablet_replica.cc:333] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.592017 17157 raft_consensus.cc:2243] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.592417 17157 raft_consensus.cc:2272] T 1e6ad8230c374b4287be43bf7cb1f75c P a5f1ac44ab684972a0d851b94e427187 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.594010 17157 tablet_replica.cc:333] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.594501 17157 raft_consensus.cc:2243] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.595254 17157 raft_consensus.cc:2272] T 14329fa0f0404c30b0da2fac0199f9a2 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.597009 17157 tablet_replica.cc:333] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.597525 17157 raft_consensus.cc:2243] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.597925 17157 raft_consensus.cc:2272] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.599802 17157 tablet_replica.cc:333] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.600351 17157 raft_consensus.cc:2243] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.601063 17157 raft_consensus.cc:2272] T b0dbdad2aed64b53a19eebfd1a1fc34f P a5f1ac44ab684972a0d851b94e427187 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.602725 17157 tablet_replica.cc:333] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187: stopping tablet replica
I20260709 12:16:11.603149 17157 raft_consensus.cc:2243] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.603621 17157 raft_consensus.cc:2272] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a5f1ac44ab684972a0d851b94e427187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.633584 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
I20260709 12:16:11.653934 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:16:11.680725 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:11.681492 17157 tablet_replica.cc:333] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.682523 17157 raft_consensus.cc:2243] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.683133 17157 raft_consensus.cc:2272] T 0a054446fbe64c51a2615a2aceb73305 P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.685832 17157 tablet_replica.cc:333] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.686561 17157 raft_consensus.cc:2243] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.687865 17157 raft_consensus.cc:2272] T 7561ca2cc72147d9ab31e3f38da55b7e P a2310c3a982443c7b2bdd928bcecb9e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.690471 17157 tablet_replica.cc:333] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.691193 17157 raft_consensus.cc:2243] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.691787 17157 raft_consensus.cc:2272] T 1e6ad8230c374b4287be43bf7cb1f75c P a2310c3a982443c7b2bdd928bcecb9e5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.694113 17157 tablet_replica.cc:333] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.694767 17157 raft_consensus.cc:2243] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.695678 17157 raft_consensus.cc:2272] T 51a0fa871a3a49b7b9720abfff3ae9b9 P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.697268 17157 tablet_replica.cc:333] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.697662 17157 raft_consensus.cc:2243] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.697962 17157 raft_consensus.cc:2272] T b0dbdad2aed64b53a19eebfd1a1fc34f P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.699505 17157 tablet_replica.cc:333] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.699939 17157 raft_consensus.cc:2243] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.700287 17157 raft_consensus.cc:2272] T 0331d62092a3425f93c053ff120a900a P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.701784 17157 tablet_replica.cc:333] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.702188 17157 raft_consensus.cc:2243] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.702526 17157 raft_consensus.cc:2272] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P a2310c3a982443c7b2bdd928bcecb9e5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.704123 17157 tablet_replica.cc:333] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.704546 17157 raft_consensus.cc:2243] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.705194 17157 raft_consensus.cc:2272] T ddbdd2bf1a034c03b48b38c4f27d11f6 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.706797 17157 tablet_replica.cc:333] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.707195 17157 raft_consensus.cc:2243] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.707598 17157 raft_consensus.cc:2272] T 0f7143273fee494e9754af0a4aafda1d P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.709105 17157 tablet_replica.cc:333] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5: stopping tablet replica
I20260709 12:16:11.709530 17157 raft_consensus.cc:2243] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.709857 17157 raft_consensus.cc:2272] T 14329fa0f0404c30b0da2fac0199f9a2 P a2310c3a982443c7b2bdd928bcecb9e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.723459 17903 consensus_peers.cc:597] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 -> Peer a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887): Couldn't send request to peer a2310c3a982443c7b2bdd928bcecb9e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:11.738379 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
I20260709 12:16:11.757850 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:16:11.781731 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:11.782303 17157 tablet_replica.cc:333] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.782874 17157 raft_consensus.cc:2243] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.783350 17157 raft_consensus.cc:2272] T 14329fa0f0404c30b0da2fac0199f9a2 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.785243 17157 tablet_replica.cc:333] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.785679 17157 raft_consensus.cc:2243] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.786391 17157 raft_consensus.cc:2272] T 0a054446fbe64c51a2615a2aceb73305 P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.787961 17157 tablet_replica.cc:333] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.788393 17157 raft_consensus.cc:2243] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.789037 17157 raft_consensus.cc:2272] T 06c9c1c9e3b343a5b11d1cdb20d38ccd P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.790568 17157 tablet_replica.cc:333] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.790988 17157 raft_consensus.cc:2243] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.791365 17157 raft_consensus.cc:2272] T 51a0fa871a3a49b7b9720abfff3ae9b9 P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.793239 17157 tablet_replica.cc:333] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.793817 17157 raft_consensus.cc:2243] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.794219 17157 raft_consensus.cc:2272] T b0dbdad2aed64b53a19eebfd1a1fc34f P 13fcc17f52a445e4b00ac27e881929c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.796345 17157 tablet_replica.cc:333] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.796828 17157 raft_consensus.cc:2243] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.797271 17157 raft_consensus.cc:2272] T 0f7143273fee494e9754af0a4aafda1d P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.797883 17903 consensus_peers.cc:597] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 -> Peer a2310c3a982443c7b2bdd928bcecb9e5 (127.16.193.66:40887): Couldn't send request to peer a2310c3a982443c7b2bdd928bcecb9e5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:40887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:11.798985 17157 tablet_replica.cc:333] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.799576 17157 raft_consensus.cc:2243] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.799971 17157 raft_consensus.cc:2272] T 7561ca2cc72147d9ab31e3f38da55b7e P 13fcc17f52a445e4b00ac27e881929c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.801556 17157 tablet_replica.cc:333] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.802017 17157 raft_consensus.cc:2243] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.802717 17157 raft_consensus.cc:2272] T 1e6ad8230c374b4287be43bf7cb1f75c P 13fcc17f52a445e4b00ac27e881929c4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.804343 17157 tablet_replica.cc:333] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.804832 17157 raft_consensus.cc:2243] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.805226 17157 raft_consensus.cc:2272] T 0331d62092a3425f93c053ff120a900a P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.806677 17157 tablet_replica.cc:333] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4: stopping tablet replica
I20260709 12:16:11.807132 17157 raft_consensus.cc:2243] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.807575 17157 raft_consensus.cc:2272] T ddbdd2bf1a034c03b48b38c4f27d11f6 P 13fcc17f52a445e4b00ac27e881929c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.853299 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:16:11.872210 17157 master.cc:562] Master@127.16.193.126:44857 shutting down...
I20260709 12:16:11.912827 17742 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 12:16:12.069552 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:12.070178 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.070531 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 00e02b5cf0e34d3d9522cccdc716b616: stopping tablet replica
I20260709 12:16:12.089232 17157 master.cc:584] Master@127.16.193.126:44857 shutdown complete.
13fcc17f52a445e4b00ac27e881929c4  1
a2310c3a982443c7b2bdd928bcecb9e5  4
a5f1ac44ab684972a0d851b94e427187  5
13fcc17f52a445e4b00ac27e881929c4  3
a2310c3a982443c7b2bdd928bcecb9e5  4
a5f1ac44ab684972a0d851b94e427187  3
13fcc17f52a445e4b00ac27e881929c4  1
a2310c3a982443c7b2bdd928bcecb9e5  1
a5f1ac44ab684972a0d851b94e427187  8
13fcc17f52a445e4b00ac27e881929c4  3
a2310c3a982443c7b2bdd928bcecb9e5  3
a5f1ac44ab684972a0d851b94e427187  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28870 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:16:12.129034 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:46027
I20260709 12:16:12.130084 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:12.135152 18173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.136631 18176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.136865 18174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.137104 17157 server_base.cc:1061] running on GCE node
I20260709 12:16:12.138113 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:12.138345 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:12.138509 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599372138494 us; error 0 us; skew 500 ppm
I20260709 12:16:12.139026 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:12.141378 17157 webserver.cc:533] Webserver started at http://127.16.193.126:38663/ using document root <none> and password file <none>
I20260709 12:16:12.141853 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:12.142051 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:12.142343 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:12.143551 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "79114be45f9144e78102be8765ae6af5"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.148052 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:16:12.151111 18181 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.151894 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:16:12.152176 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "79114be45f9144e78102be8765ae6af5"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.152467 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:12.164280 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:12.165462 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:12.202514 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:46027
I20260709 12:16:12.202602 18232 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:46027 every 8 connection(s)
I20260709 12:16:12.206284 18233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.216077 18233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Bootstrap starting.
I20260709 12:16:12.220717 18233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:12.224632 18233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: No bootstrap required, opened a new log
I20260709 12:16:12.226658 18233 raft_consensus.cc:359] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER }
I20260709 12:16:12.227047 18233 raft_consensus.cc:385] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:12.227339 18233 raft_consensus.cc:740] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79114be45f9144e78102be8765ae6af5, State: Initialized, Role: FOLLOWER
I20260709 12:16:12.227916 18233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER }
I20260709 12:16:12.228370 18233 raft_consensus.cc:399] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:12.228586 18233 raft_consensus.cc:493] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:12.228878 18233 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:12.232951 18233 raft_consensus.cc:515] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER }
I20260709 12:16:12.233451 18233 leader_election.cc:304] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79114be45f9144e78102be8765ae6af5; no voters: 
I20260709 12:16:12.234643 18233 leader_election.cc:290] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:12.234968 18236 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:12.236245 18236 raft_consensus.cc:697] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 1 LEADER]: Becoming Leader. State: Replica: 79114be45f9144e78102be8765ae6af5, State: Running, Role: LEADER
I20260709 12:16:12.236932 18236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER }
I20260709 12:16:12.237591 18233 sys_catalog.cc:565] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:12.242028 18238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79114be45f9144e78102be8765ae6af5. Latest consensus state: current_term: 1 leader_uuid: "79114be45f9144e78102be8765ae6af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER } }
I20260709 12:16:12.242686 18238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:12.243848 18237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79114be45f9144e78102be8765ae6af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79114be45f9144e78102be8765ae6af5" member_type: VOTER } }
I20260709 12:16:12.244412 18237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:12.244683 18244 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:12.249670 18244 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:12.255116 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:12.259176 18244 catalog_manager.cc:1383] Generated new cluster ID: 21f27c97fb6d41959017592263f707c2
I20260709 12:16:12.259476 18244 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:12.278251 18244 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:12.279453 18244 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:12.300948 18244 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Generated new TSK 0
I20260709 12:16:12.301540 18244 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:12.322209 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:12.327907 18254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.329113 18255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.330536 18257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.330824 17157 server_base.cc:1061] running on GCE node
I20260709 12:16:12.331822 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:12.331996 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:12.332114 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599372332103 us; error 0 us; skew 500 ppm
I20260709 12:16:12.332576 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:12.334648 17157 webserver.cc:533] Webserver started at http://127.16.193.65:45783/ using document root <none> and password file <none>
I20260709 12:16:12.335069 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:12.335217 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:12.335525 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:12.336575 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "e23c0ce8d89940da8eac6a6002e349c5"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.340962 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:16:12.343979 18262 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.344681 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:16:12.344985 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "e23c0ce8d89940da8eac6a6002e349c5"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.345269 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:12.356748 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:12.357950 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:12.359431 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:12.361586 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:12.361809 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.362069 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:12.362267 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.400981 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:37771
I20260709 12:16:12.401067 18324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:37771 every 8 connection(s)
I20260709 12:16:12.405704 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:12.412722 18329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.413468 18330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.416999 17157 server_base.cc:1061] running on GCE node
W20260709 12:16:12.417351 18333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.418746 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:12.419031 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:12.419288 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599372419267 us; error 0 us; skew 500 ppm
I20260709 12:16:12.419868 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:12.421952 18325 heartbeater.cc:344] Connected to a master server at 127.16.193.126:46027
I20260709 12:16:12.422351 18325 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:12.422753 17157 webserver.cc:533] Webserver started at http://127.16.193.66:35643/ using document root <none> and password file <none>
I20260709 12:16:12.423281 18325 heartbeater.cc:507] Master 127.16.193.126:46027 requested a full tablet report, sending...
I20260709 12:16:12.423386 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:12.423703 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:12.424036 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:12.425300 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "3da7ad8fa20a4591963b660e4c848e7f"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.425633 18198 ts_manager.cc:194] Registered new tserver with Master: e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:12.427984 18198 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46430
I20260709 12:16:12.430732 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:16:12.433887 18337 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.434571 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:12.434868 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "3da7ad8fa20a4591963b660e4c848e7f"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.435146 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:12.453130 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:12.454352 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:12.455778 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:12.457871 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:12.458061 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.458317 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:12.458472 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.495942 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:44577
I20260709 12:16:12.496034 18399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:44577 every 8 connection(s)
I20260709 12:16:12.500514 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:12.507256 18403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:12.509686 18404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.513404 18400 heartbeater.cc:344] Connected to a master server at 127.16.193.126:46027
I20260709 12:16:12.513477 17157 server_base.cc:1061] running on GCE node
W20260709 12:16:12.513970 18406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:12.513989 18400 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:12.515089 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:16:12.515146 18400 heartbeater.cc:507] Master 127.16.193.126:46027 requested a full tablet report, sending...
W20260709 12:16:12.515465 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:12.515678 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599372515660 us; error 0 us; skew 500 ppm
I20260709 12:16:12.516369 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:12.517359 18198 ts_manager.cc:194] Registered new tserver with Master: 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:12.518893 18198 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46436
I20260709 12:16:12.519527 17157 webserver.cc:533] Webserver started at http://127.16.193.67:34469/ using document root <none> and password file <none>
I20260709 12:16:12.520174 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:12.520375 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:12.520663 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:12.521749 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "177254018ba7448aa2b9110c9c9be9e3"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.525883 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:16:12.528868 18411 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.529531 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:12.529825 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "177254018ba7448aa2b9110c9c9be9e3"
format_stamp: "Formatted at 2026-07-09 12:16:12 on dist-test-slave-n640"
I20260709 12:16:12.530086 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:12.539697 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:12.540777 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:12.542095 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:12.544237 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:12.544463 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.544693 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:12.544868 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:12.582988 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:40227
I20260709 12:16:12.583074 18473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:40227 every 8 connection(s)
I20260709 12:16:12.596334 18474 heartbeater.cc:344] Connected to a master server at 127.16.193.126:46027
I20260709 12:16:12.596697 18474 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:12.597388 18474 heartbeater.cc:507] Master 127.16.193.126:46027 requested a full tablet report, sending...
I20260709 12:16:12.599411 18198 ts_manager.cc:194] Registered new tserver with Master: 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:12.600267 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014390483s
I20260709 12:16:12.601089 18198 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46444
I20260709 12:16:12.626981 18198 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46458:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:12.633149 18198 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:12.875799 18362 tablet_service.cc:1511] Processing CreateTablet for tablet cc5f8db079f74a52b906175ad2c4cb56 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:12.876856 18361 tablet_service.cc:1511] Processing CreateTablet for tablet dc49a9d4c95c46f38ca4a279c913204d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:12.878119 18362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc5f8db079f74a52b906175ad2c4cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.879177 18290 tablet_service.cc:1511] Processing CreateTablet for tablet e9decd1c7d7f475b8890a8519847dca5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:12.879465 18360 tablet_service.cc:1511] Processing CreateTablet for tablet 4b135a527cce4b14b789d23618d411b0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:12.880681 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9decd1c7d7f475b8890a8519847dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.880522 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 0931a54c049145dfad672173c0b29a40 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:12.882105 18359 tablet_service.cc:1511] Processing CreateTablet for tablet 3153d259c2254b2d9bdc71d8322cafc0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:12.884472 18289 tablet_service.cc:1511] Processing CreateTablet for tablet d266f68528964b1981ffe1df49861b08 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:12.884538 18365 tablet_service.cc:1511] Processing CreateTablet for tablet e9decd1c7d7f475b8890a8519847dca5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:12.885108 18358 tablet_service.cc:1511] Processing CreateTablet for tablet 6021c985b4e44fe880350a321132a9b5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:12.885782 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d266f68528964b1981ffe1df49861b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.888051 18288 tablet_service.cc:1511] Processing CreateTablet for tablet 0931a54c049145dfad672173c0b29a40 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:12.889436 18288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0931a54c049145dfad672173c0b29a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.889385 18357 tablet_service.cc:1511] Processing CreateTablet for tablet adea31c273544d04b7f16f9b9a3ae610 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:12.882930 18364 tablet_service.cc:1511] Processing CreateTablet for tablet d266f68528964b1981ffe1df49861b08 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:12.892093 18439 tablet_service.cc:1511] Processing CreateTablet for tablet e9decd1c7d7f475b8890a8519847dca5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:12.892468 18286 tablet_service.cc:1511] Processing CreateTablet for tablet dc49a9d4c95c46f38ca4a279c913204d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:12.893643 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9decd1c7d7f475b8890a8519847dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.893821 18286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc49a9d4c95c46f38ca4a279c913204d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.896448 18285 tablet_service.cc:1511] Processing CreateTablet for tablet 4b135a527cce4b14b789d23618d411b0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:12.896600 18434 tablet_service.cc:1511] Processing CreateTablet for tablet 4b135a527cce4b14b789d23618d411b0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:12.897738 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b135a527cce4b14b789d23618d411b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.899304 18431 tablet_service.cc:1511] Processing CreateTablet for tablet adea31c273544d04b7f16f9b9a3ae610 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:12.899833 18438 tablet_service.cc:1511] Processing CreateTablet for tablet d266f68528964b1981ffe1df49861b08 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:12.901185 18430 tablet_service.cc:1511] Processing CreateTablet for tablet 1686d6e6b8ae45aab8c5d9d916f8485d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:12.901897 18434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b135a527cce4b14b789d23618d411b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.917900 18284 tablet_service.cc:1511] Processing CreateTablet for tablet 3153d259c2254b2d9bdc71d8322cafc0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:12.901669 18437 tablet_service.cc:1511] Processing CreateTablet for tablet 0931a54c049145dfad672173c0b29a40 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:12.902452 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b135a527cce4b14b789d23618d411b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.903498 18435 tablet_service.cc:1511] Processing CreateTablet for tablet dc49a9d4c95c46f38ca4a279c913204d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:12.905282 18433 tablet_service.cc:1511] Processing CreateTablet for tablet 3153d259c2254b2d9bdc71d8322cafc0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:12.906503 18356 tablet_service.cc:1511] Processing CreateTablet for tablet 1686d6e6b8ae45aab8c5d9d916f8485d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:12.907058 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 6021c985b4e44fe880350a321132a9b5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:12.909158 18436 tablet_service.cc:1511] Processing CreateTablet for tablet cc5f8db079f74a52b906175ad2c4cb56 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:12.891121 18287 tablet_service.cc:1511] Processing CreateTablet for tablet cc5f8db079f74a52b906175ad2c4cb56 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:12.919216 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153d259c2254b2d9bdc71d8322cafc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.935904 18287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc5f8db079f74a52b906175ad2c4cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.920327 18437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0931a54c049145dfad672173c0b29a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.940591 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6021c985b4e44fe880350a321132a9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.921739 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9decd1c7d7f475b8890a8519847dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.943544 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc49a9d4c95c46f38ca4a279c913204d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.924402 18282 tablet_service.cc:1511] Processing CreateTablet for tablet adea31c273544d04b7f16f9b9a3ae610 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:12.923112 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 6021c985b4e44fe880350a321132a9b5 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:12.944690 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adea31c273544d04b7f16f9b9a3ae610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.943559 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1686d6e6b8ae45aab8c5d9d916f8485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.945960 18357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adea31c273544d04b7f16f9b9a3ae610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.925267 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 1686d6e6b8ae45aab8c5d9d916f8485d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:12.946597 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6021c985b4e44fe880350a321132a9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.947098 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d266f68528964b1981ffe1df49861b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.947324 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1686d6e6b8ae45aab8c5d9d916f8485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.948318 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0931a54c049145dfad672173c0b29a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.951653 18359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153d259c2254b2d9bdc71d8322cafc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.953099 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc49a9d4c95c46f38ca4a279c913204d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.944763 18433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153d259c2254b2d9bdc71d8322cafc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.958720 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1686d6e6b8ae45aab8c5d9d916f8485d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.959712 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6021c985b4e44fe880350a321132a9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.960803 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc5f8db079f74a52b906175ad2c4cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.962177 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adea31c273544d04b7f16f9b9a3ae610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:12.963536 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d266f68528964b1981ffe1df49861b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.060158 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 85f32970b7374dcf9b9bac47791c71e3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:13.061573 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f32970b7374dcf9b9bac47791c71e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.080487 18490 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:13.091962 18435 tablet_service.cc:1511] Processing CreateTablet for tablet 85f32970b7374dcf9b9bac47791c71e3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:13.093291 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f32970b7374dcf9b9bac47791c71e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.078743 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 01de3fbc59f243ba8a35edaa5fcfe86f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:13.107285 18285 tablet_service.cc:1511] Processing CreateTablet for tablet 35973fcee55042b284c6f62e2681c304 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:13.109123 18360 tablet_service.cc:1511] Processing CreateTablet for tablet 01de3fbc59f243ba8a35edaa5fcfe86f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:13.110376 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01de3fbc59f243ba8a35edaa5fcfe86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.111967 18490 tablet_bootstrap.cc:654] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.120370 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 01de3fbc59f243ba8a35edaa5fcfe86f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:13.122020 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01de3fbc59f243ba8a35edaa5fcfe86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.127823 18287 tablet_service.cc:1511] Processing CreateTablet for tablet 389aa517d744435bb043ff8c52d86c2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:13.136370 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 35973fcee55042b284c6f62e2681c304 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:13.137789 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35973fcee55042b284c6f62e2681c304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.149324 18357 tablet_service.cc:1511] Processing CreateTablet for tablet 35973fcee55042b284c6f62e2681c304 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:13.150650 18357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35973fcee55042b284c6f62e2681c304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.151014 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 85f32970b7374dcf9b9bac47791c71e3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:13.152305 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85f32970b7374dcf9b9bac47791c71e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.154769 18362 tablet_service.cc:1511] Processing CreateTablet for tablet 389aa517d744435bb043ff8c52d86c2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:13.163744 18284 tablet_service.cc:1511] Processing CreateTablet for tablet ca383fa1243547c3b32dfbd6857a3fba (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:13.165354 18431 tablet_service.cc:1511] Processing CreateTablet for tablet 389aa517d744435bb043ff8c52d86c2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:13.166565 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389aa517d744435bb043ff8c52d86c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.167415 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01de3fbc59f243ba8a35edaa5fcfe86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.167719 18286 tablet_service.cc:1511] Processing CreateTablet for tablet c5398d6474dd4eed946f45af9214825c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:13.169023 18286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5398d6474dd4eed946f45af9214825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.176843 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 72e8da9e5fd9449f93381781e125fada (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:13.177112 18289 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5d2fad2696483cae18eab3ba337ec9 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:13.178694 18492 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:13.179171 18436 tablet_service.cc:1511] Processing CreateTablet for tablet ca383fa1243547c3b32dfbd6857a3fba (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:13.180491 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca383fa1243547c3b32dfbd6857a3fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.181641 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca383fa1243547c3b32dfbd6857a3fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.183686 18491 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:13.184360 18364 tablet_service.cc:1511] Processing CreateTablet for tablet ca383fa1243547c3b32dfbd6857a3fba (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:13.185437 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5d2fad2696483cae18eab3ba337ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.185853 18358 tablet_service.cc:1511] Processing CreateTablet for tablet 72e8da9e5fd9449f93381781e125fada (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:13.187582 18287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389aa517d744435bb043ff8c52d86c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.188944 18439 tablet_service.cc:1511] Processing CreateTablet for tablet c5398d6474dd4eed946f45af9214825c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:13.190163 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5398d6474dd4eed946f45af9214825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.191808 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e8da9e5fd9449f93381781e125fada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.193946 18356 tablet_service.cc:1511] Processing CreateTablet for tablet c5398d6474dd4eed946f45af9214825c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:13.195062 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35973fcee55042b284c6f62e2681c304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.204231 18282 tablet_service.cc:1511] Processing CreateTablet for tablet f53e315b86a94adb89240687e1a622ed (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:13.205927 18437 tablet_service.cc:1511] Processing CreateTablet for tablet 72e8da9e5fd9449f93381781e125fada (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:13.207262 18437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e8da9e5fd9449f93381781e125fada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.213002 18289 tablet_service.cc:1511] Processing CreateTablet for tablet ad852e5220a14f4898d2f069a5fb4a63 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:13.214071 18434 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5d2fad2696483cae18eab3ba337ec9 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:13.214512 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 548b719daf794fb881c5606ebec2abb3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:13.217978 18434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5d2fad2696483cae18eab3ba337ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.210709 18288 tablet_service.cc:1511] Processing CreateTablet for tablet d8bf6d66a261490caa69a80b24ea015f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:13.219808 18362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389aa517d744435bb043ff8c52d86c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.222565 18491 tablet_bootstrap.cc:654] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.224412 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e8da9e5fd9449f93381781e125fada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.228500 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca383fa1243547c3b32dfbd6857a3fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.233829 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5d2fad2696483cae18eab3ba337ec9 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:13.239181 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5d2fad2696483cae18eab3ba337ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.244238 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5398d6474dd4eed946f45af9214825c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:13.251438 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:13.254316 18492 tablet_bootstrap.cc:654] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.255308 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53e315b86a94adb89240687e1a622ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.255942 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad852e5220a14f4898d2f069a5fb4a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.255782 18433 tablet_service.cc:1511] Processing CreateTablet for tablet 548b719daf794fb881c5606ebec2abb3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:13.257248 18433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b719daf794fb881c5606ebec2abb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.257745 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b719daf794fb881c5606ebec2abb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.261142 18281 tablet_service.cc:1511] Processing CreateTablet for tablet b81b910b60b0422f9db4c0164608fcbb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:13.264263 18432 tablet_service.cc:1511] Processing CreateTablet for tablet d8bf6d66a261490caa69a80b24ea015f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:13.267943 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bf6d66a261490caa69a80b24ea015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.267729 18430 tablet_service.cc:1511] Processing CreateTablet for tablet ad852e5220a14f4898d2f069a5fb4a63 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:13.272758 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad852e5220a14f4898d2f069a5fb4a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.268397 18288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bf6d66a261490caa69a80b24ea015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.266026 18359 tablet_service.cc:1511] Processing CreateTablet for tablet d8bf6d66a261490caa69a80b24ea015f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:13.259701 18365 tablet_service.cc:1511] Processing CreateTablet for tablet 548b719daf794fb881c5606ebec2abb3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:13.285714 18490 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:13.286228 18490 ts_tablet_manager.cc:1403] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.206s	user 0.003s	sys 0.010s
I20260709 12:16:13.289666 18490 raft_consensus.cc:359] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.290863 18490 raft_consensus.cc:385] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.291209 18490 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.291544 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81b910b60b0422f9db4c0164608fcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.295866 18359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bf6d66a261490caa69a80b24ea015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.297384 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b719daf794fb881c5606ebec2abb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.303519 18490 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.292820 18436 tablet_service.cc:1511] Processing CreateTablet for tablet b81b910b60b0422f9db4c0164608fcbb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:13.309132 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81b910b60b0422f9db4c0164608fcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.318114 18287 tablet_service.cc:1511] Processing CreateTablet for tablet c1f155085cb647ff957b468384227872 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:13.325956 18431 tablet_service.cc:1511] Processing CreateTablet for tablet f53e315b86a94adb89240687e1a622ed (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:13.339375 18287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f155085cb647ff957b468384227872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.341140 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53e315b86a94adb89240687e1a622ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.350795 18400 heartbeater.cc:499] Master 127.16.193.126:46027 was elected leader, sending a full tablet report...
I20260709 12:16:13.356652 18490 ts_tablet_manager.cc:1434] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.004s
I20260709 12:16:13.357801 18490 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:13.338832 18439 tablet_service.cc:1511] Processing CreateTablet for tablet c1f155085cb647ff957b468384227872 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:13.360361 18362 tablet_service.cc:1511] Processing CreateTablet for tablet ad852e5220a14f4898d2f069a5fb4a63 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:13.364344 18435 tablet_service.cc:1511] Processing CreateTablet for tablet 53c08deb8e86480a878b17c55d6bc089 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:13.365026 18360 tablet_service.cc:1511] Processing CreateTablet for tablet b81b910b60b0422f9db4c0164608fcbb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:13.365619 18364 tablet_service.cc:1511] Processing CreateTablet for tablet f53e315b86a94adb89240687e1a622ed (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:13.379617 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f155085cb647ff957b468384227872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.389789 18492 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:13.390260 18492 ts_tablet_manager.cc:1403] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.212s	user 0.009s	sys 0.010s
I20260709 12:16:13.390142 18284 tablet_service.cc:1511] Processing CreateTablet for tablet 53c08deb8e86480a878b17c55d6bc089 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:13.390623 18434 tablet_service.cc:1511] Processing CreateTablet for tablet b9f2e3c205c14e43b532c52bc7fa4c90 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:13.371333 18290 tablet_service.cc:1511] Processing CreateTablet for tablet fafcfc4614304a81b0eadc8f445bfdb8 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:13.384603 18358 tablet_service.cc:1511] Processing CreateTablet for tablet c1f155085cb647ff957b468384227872 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:13.398151 18491 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:13.398307 18356 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4622d18b474b659ebea19e9aad0d05 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:13.398744 18491 ts_tablet_manager.cc:1403] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.223s	user 0.003s	sys 0.019s
I20260709 12:16:13.401217 18490 tablet_bootstrap.cc:654] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.397795 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4622d18b474b659ebea19e9aad0d05 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:13.401475 18437 tablet_service.cc:1511] Processing CreateTablet for tablet a971a418238d45d288b49960c0abeea0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:13.403455 18285 tablet_service.cc:1511] Processing CreateTablet for tablet b9f2e3c205c14e43b532c52bc7fa4c90 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:13.407781 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c08deb8e86480a878b17c55d6bc089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.413000 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b81b910b60b0422f9db4c0164608fcbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.413883 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53e315b86a94adb89240687e1a622ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.415982 18492 raft_consensus.cc:359] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.416680 18492 raft_consensus.cc:385] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.416961 18492 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.417467 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4622d18b474b659ebea19e9aad0d05 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:13.417850 18492 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.418690 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4622d18b474b659ebea19e9aad0d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.419660 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f155085cb647ff957b468384227872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.419739 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fafcfc4614304a81b0eadc8f445bfdb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.424914 18289 tablet_service.cc:1511] Processing CreateTablet for tablet a971a418238d45d288b49960c0abeea0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:13.402748 18491 raft_consensus.cc:359] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.425927 18491 raft_consensus.cc:385] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.426455 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a971a418238d45d288b49960c0abeea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.427377 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 1a92f90eb0424e3d870764a2a328e508 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:13.393818 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 7d788f8182ca4b13bddd2acdf511873f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:13.428776 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a92f90eb0424e3d870764a2a328e508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.428929 18362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad852e5220a14f4898d2f069a5fb4a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.429540 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4622d18b474b659ebea19e9aad0d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.430173 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c08deb8e86480a878b17c55d6bc089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.419786 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 53c08deb8e86480a878b17c55d6bc089 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:13.431538 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c08deb8e86480a878b17c55d6bc089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.428965 18365 tablet_service.cc:1511] Processing CreateTablet for tablet b9f2e3c205c14e43b532c52bc7fa4c90 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:13.431711 18325 heartbeater.cc:499] Master 127.16.193.126:46027 was elected leader, sending a full tablet report...
I20260709 12:16:13.433295 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f2e3c205c14e43b532c52bc7fa4c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.434988 18437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a971a418238d45d288b49960c0abeea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.437566 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d788f8182ca4b13bddd2acdf511873f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.441968 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f2e3c205c14e43b532c52bc7fa4c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.443974 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 7d788f8182ca4b13bddd2acdf511873f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:13.445168 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d788f8182ca4b13bddd2acdf511873f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.459528 18434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f2e3c205c14e43b532c52bc7fa4c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.460011 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4622d18b474b659ebea19e9aad0d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.472607 18286 tablet_service.cc:1511] Processing CreateTablet for tablet 7d788f8182ca4b13bddd2acdf511873f (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:13.475301 18288 tablet_service.cc:1511] Processing CreateTablet for tablet e98a95c8e3c14bcfa51b054f4829293b (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:13.477502 18282 tablet_service.cc:1511] Processing CreateTablet for tablet 57dbd4858e8f41a2af67d6de6eae9928 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:13.482839 18357 tablet_service.cc:1511] Processing CreateTablet for tablet a971a418238d45d288b49960c0abeea0 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:13.426457 18491 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.484221 18357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a971a418238d45d288b49960c0abeea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.450750 18433 tablet_service.cc:1511] Processing CreateTablet for tablet fafcfc4614304a81b0eadc8f445bfdb8 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:13.487537 18491 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.488162 18433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fafcfc4614304a81b0eadc8f445bfdb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:13.500731 18401 tablet.cc:2406] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:13.512501 18288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98a95c8e3c14bcfa51b054f4829293b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.515909 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dbd4858e8f41a2af67d6de6eae9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.517103 18287 tablet_service.cc:1511] Processing CreateTablet for tablet 977c0af9c8e94119b3d18a0ee7401458 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:13.518388 18287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977c0af9c8e94119b3d18a0ee7401458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.528378 18364 tablet_service.cc:1511] Processing CreateTablet for tablet fafcfc4614304a81b0eadc8f445bfdb8 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:13.529633 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fafcfc4614304a81b0eadc8f445bfdb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.531394 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 57dbd4858e8f41a2af67d6de6eae9928 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:13.533366 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dbd4858e8f41a2af67d6de6eae9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.535727 18286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d788f8182ca4b13bddd2acdf511873f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.539165 18360 tablet_service.cc:1511] Processing CreateTablet for tablet 1a92f90eb0424e3d870764a2a328e508 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:13.547657 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a92f90eb0424e3d870764a2a328e508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.550987 18491 ts_tablet_manager.cc:1434] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.152s	user 0.009s	sys 0.001s
I20260709 12:16:13.544173 18365 tablet_service.cc:1511] Processing CreateTablet for tablet 57dbd4858e8f41a2af67d6de6eae9928 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:13.553973 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dbd4858e8f41a2af67d6de6eae9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.554961 18491 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:13.556017 18430 tablet_service.cc:1511] Processing CreateTablet for tablet 1a92f90eb0424e3d870764a2a328e508 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:13.557305 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a92f90eb0424e3d870764a2a328e508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.526924 18436 tablet_service.cc:1511] Processing CreateTablet for tablet e98a95c8e3c14bcfa51b054f4829293b (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:13.560236 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98a95c8e3c14bcfa51b054f4829293b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.561028 18358 tablet_service.cc:1511] Processing CreateTablet for tablet e98a95c8e3c14bcfa51b054f4829293b (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:13.562536 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98a95c8e3c14bcfa51b054f4829293b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.574868 18359 tablet_service.cc:1511] Processing CreateTablet for tablet 977c0af9c8e94119b3d18a0ee7401458 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:13.576912 18437 tablet_service.cc:1511] Processing CreateTablet for tablet 977c0af9c8e94119b3d18a0ee7401458 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:13.578269 18437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977c0af9c8e94119b3d18a0ee7401458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.582499 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 353cbd244a354830a4e903a3b1be2b89 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:13.594437 18431 tablet_service.cc:1511] Processing CreateTablet for tablet 353cbd244a354830a4e903a3b1be2b89 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:13.596303 18439 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbd14f2572940d797f0297b94365af4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:13.604764 18359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977c0af9c8e94119b3d18a0ee7401458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.606212 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353cbd244a354830a4e903a3b1be2b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.606426 18474 heartbeater.cc:499] Master 127.16.193.126:46027 was elected leader, sending a full tablet report...
I20260709 12:16:13.607856 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 353cbd244a354830a4e903a3b1be2b89 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:13.608924 18492 ts_tablet_manager.cc:1434] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.218s	user 0.010s	sys 0.007s
I20260709 12:16:13.609078 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353cbd244a354830a4e903a3b1be2b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.624506 18362 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbd14f2572940d797f0297b94365af4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:13.625150 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 5461f19f144642beb790dcf7f1515a2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:13.629747 18498 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.619129 18289 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbd14f2572940d797f0297b94365af4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:13.630285 18498 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.631847 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbd14f2572940d797f0297b94365af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.633198 18498 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:13.642136 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461f19f144642beb790dcf7f1515a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.669345 18490 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:13.669869 18490 ts_tablet_manager.cc:1403] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.312s	user 0.019s	sys 0.027s
I20260709 12:16:13.683916 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353cbd244a354830a4e903a3b1be2b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.684188 18492 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:13.689718 18436 tablet_service.cc:1511] Processing CreateTablet for tablet 5461f19f144642beb790dcf7f1515a2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:13.691097 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461f19f144642beb790dcf7f1515a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.692929 18283 tablet_service.cc:1511] Processing CreateTablet for tablet dc5742413fd242b88ca8c4668e6b8144 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:13.694350 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5742413fd242b88ca8c4668e6b8144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.696770 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4d1763576e4c49849611360f37665a (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:13.698060 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4d1763576e4c49849611360f37665a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.700719 18435 tablet_service.cc:1511] Processing CreateTablet for tablet dc5742413fd242b88ca8c4668e6b8144 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:13.701972 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5742413fd242b88ca8c4668e6b8144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.704905 18438 tablet_service.cc:1511] Processing CreateTablet for tablet c749b52c1c8c44b78cee98b4b4d2ef29 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:13.706116 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c749b52c1c8c44b78cee98b4b4d2ef29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.705957 18288 tablet_service.cc:1511] Processing CreateTablet for tablet 80bc05a330c64471aaa556af75bea031 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:13.707537 18288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80bc05a330c64471aaa556af75bea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.707757 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbd14f2572940d797f0297b94365af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.708758 18434 tablet_service.cc:1511] Processing CreateTablet for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:13.709931 18434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.716173 18284 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4d1763576e4c49849611360f37665a (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:13.717338 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4d1763576e4c49849611360f37665a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.719969 18433 tablet_service.cc:1511] Processing CreateTablet for tablet 80bc05a330c64471aaa556af75bea031 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:13.721311 18433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80bc05a330c64471aaa556af75bea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.723299 18362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbd14f2572940d797f0297b94365af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.724212 18356 tablet_service.cc:1511] Processing CreateTablet for tablet 5461f19f144642beb790dcf7f1515a2d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:13.725430 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461f19f144642beb790dcf7f1515a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.729336 18287 tablet_service.cc:1511] Processing CreateTablet for tablet c749b52c1c8c44b78cee98b4b4d2ef29 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:13.731088 18287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c749b52c1c8c44b78cee98b4b4d2ef29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.760937 18491 tablet_bootstrap.cc:654] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.765588 18357 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4d1763576e4c49849611360f37665a (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:13.677711 18490 raft_consensus.cc:359] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.778187 18492 tablet_bootstrap.cc:654] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.781611 18364 tablet_service.cc:1511] Processing CreateTablet for tablet dc5742413fd242b88ca8c4668e6b8144 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:13.778761 18490 raft_consensus.cc:385] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.786603 18490 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.779470 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 80bc05a330c64471aaa556af75bea031 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:13.790410 18490 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.794833 18490 ts_tablet_manager.cc:1434] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.125s	user 0.006s	sys 0.003s
I20260709 12:16:13.800843 18358 tablet_service.cc:1511] Processing CreateTablet for tablet c749b52c1c8c44b78cee98b4b4d2ef29 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:13.802318 18430 tablet_service.cc:1511] Processing CreateTablet for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:13.804088 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.804318 18434 tablet_service.cc:1511] Processing CreateTablet for tablet b4688e522b8b48f7bb98e0832680f9e2 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:13.805548 18434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4688e522b8b48f7bb98e0832680f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.813199 18285 tablet_service.cc:1511] Processing CreateTablet for tablet b4688e522b8b48f7bb98e0832680f9e2 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:13.814426 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4688e522b8b48f7bb98e0832680f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.775182 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:13.827134 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.839258 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc8b80684174ecb8c933560e8143c49 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:13.840631 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc8b80684174ecb8c933560e8143c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.791118 18357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4d1763576e4c49849611360f37665a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.847664 18495 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.848446 18495 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.849118 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:13.847966 18490 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:13.863329 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80bc05a330c64471aaa556af75bea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.868340 18360 tablet_service.cc:1511] Processing CreateTablet for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:13.869558 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:13.872457 18412 leader_election.cc:343] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:13.877496 18282 tablet_service.cc:1511] Processing CreateTablet for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:13.881098 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c749b52c1c8c44b78cee98b4b4d2ef29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.888417 18495 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:13.900640 18283 tablet_service.cc:1511] Processing CreateTablet for tablet 8903d70080d24236be955433a92791c7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:13.900787 18365 tablet_service.cc:1511] Processing CreateTablet for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:13.904735 18286 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab0a3f9b34149918a57aec5d20b4884 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:13.922221 18363 tablet_service.cc:1511] Processing CreateTablet for tablet b4688e522b8b48f7bb98e0832680f9e2 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:13.925215 18499 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.925853 18499 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.932085 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 0071c33745ae42e48eca14d86aa5fac6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:13.913463 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5742413fd242b88ca8c4668e6b8144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.945410 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.948026 18490 tablet_bootstrap.cc:654] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.938830 18499 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:13.962908 18492 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:13.963459 18492 ts_tablet_manager.cc:1403] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.280s	user 0.036s	sys 0.038s
I20260709 12:16:13.964953 18439 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab0a3f9b34149918a57aec5d20b4884 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:13.966213 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab0a3f9b34149918a57aec5d20b4884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.966431 18492 raft_consensus.cc:359] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.967689 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0071c33745ae42e48eca14d86aa5fac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.963045 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 8903d70080d24236be955433a92791c7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:13.968653 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8903d70080d24236be955433a92791c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.967131 18492 raft_consensus.cc:385] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.976282 18492 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.978358 18286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab0a3f9b34149918a57aec5d20b4884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.978855 18437 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc8b80684174ecb8c933560e8143c49 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:13.979220 18289 tablet_service.cc:1511] Processing CreateTablet for tablet 0071c33745ae42e48eca14d86aa5fac6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:13.980806 18283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8903d70080d24236be955433a92791c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.983209 18289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0071c33745ae42e48eca14d86aa5fac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.981232 18492 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:13.996335 18491 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:13.996845 18491 ts_tablet_manager.cc:1403] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.442s	user 0.030s	sys 0.015s
I20260709 12:16:13.999743 18491 raft_consensus.cc:359] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.000443 18491 raft_consensus.cc:385] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.000746 18491 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.001432 18491 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.008759 18437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc8b80684174ecb8c933560e8143c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.012676 18436 tablet_service.cc:1511] Processing CreateTablet for tablet 37ef0235d410491ab47198988075f6e4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:14.014073 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ef0235d410491ab47198988075f6e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.016058 18492 ts_tablet_manager.cc:1434] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.052s	user 0.002s	sys 0.004s
I20260709 12:16:14.017028 18492 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.024636 18433 tablet_service.cc:1511] Processing CreateTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:14.025141 18431 tablet_service.cc:1511] Processing CreateTablet for tablet 9934c84956844d20a6258c5dca3833c6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:14.025950 18433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea246fa8cc7e4edbb136da2ee92fb1d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.034732 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 37ef0235d410491ab47198988075f6e4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:14.035776 18284 tablet_service.cc:1511] Processing CreateTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:14.036332 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2ee63a449648bfb064fadee177cadb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:14.036166 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ef0235d410491ab47198988075f6e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.037748 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2ee63a449648bfb064fadee177cadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.038396 18495 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.038858 18495 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.039887 18282 tablet_service.cc:1511] Processing CreateTablet for tablet 9934c84956844d20a6258c5dca3833c6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:13.980907 18356 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc8b80684174ecb8c933560e8143c49 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:14.040937 18495 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.041127 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9934c84956844d20a6258c5dca3833c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.041718 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea246fa8cc7e4edbb136da2ee92fb1d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.046090 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:14.047854 18415 leader_election.cc:343] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.048307 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.048456 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.049528 18500 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.049853 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:14.055763 18338 leader_election.cc:343] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:14.056445 18338 leader_election.cc:343] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.029879 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.059386 18491 ts_tablet_manager.cc:1434] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.002s
I20260709 12:16:14.062733 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc8b80684174ecb8c933560e8143c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.064339 18491 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.065958 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4688e522b8b48f7bb98e0832680f9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.076630 18362 tablet_service.cc:1511] Processing CreateTablet for tablet 6ab0a3f9b34149918a57aec5d20b4884 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:14.078032 18362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab0a3f9b34149918a57aec5d20b4884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.031381 18492 tablet_bootstrap.cc:654] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.093510 18359 tablet_service.cc:1511] Processing CreateTablet for tablet 8903d70080d24236be955433a92791c7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:14.094954 18359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8903d70080d24236be955433a92791c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.095329 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9934c84956844d20a6258c5dca3833c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.101907 18282 tablet_service.cc:1511] Processing CreateTablet for tablet e7d0369b4aea4afb960bf859e9b7738c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:14.108229 18282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d0369b4aea4afb960bf859e9b7738c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.108389 18500 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.108862 18500 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.109373 18360 tablet_service.cc:1511] Processing CreateTablet for tablet 37ef0235d410491ab47198988075f6e4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:14.110649 18360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ef0235d410491ab47198988075f6e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.112370 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:14.113828 18412 leader_election.cc:343] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.117795 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.118161 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.118494 18448 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.120301 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.121129 18495 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.121510 18495 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.121846 18495 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.124181 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.111092 18500 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:16:14.127769 18339 leader_election.cc:343] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.128214 18339 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.128454 18435 tablet_service.cc:1511] Processing CreateTablet for tablet e7d0369b4aea4afb960bf859e9b7738c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:14.132838 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d0369b4aea4afb960bf859e9b7738c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.158247 18285 tablet_service.cc:1511] Processing CreateTablet for tablet 67a1783f49244bd5ba98721205d1ff6e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:14.159739 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a1783f49244bd5ba98721205d1ff6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.168690 18432 tablet_service.cc:1511] Processing CreateTablet for tablet 67a1783f49244bd5ba98721205d1ff6e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:14.168902 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 0071c33745ae42e48eca14d86aa5fac6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:14.170114 18432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a1783f49244bd5ba98721205d1ff6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.170746 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0071c33745ae42e48eca14d86aa5fac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.176260 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:14.185793 18266 leader_election.cc:343] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.187608 18490 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.188136 18490 ts_tablet_manager.cc:1403] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.342s	user 0.020s	sys 0.040s
I20260709 12:16:14.189561 18358 tablet_service.cc:1511] Processing CreateTablet for tablet 9934c84956844d20a6258c5dca3833c6 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:14.191088 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9934c84956844d20a6258c5dca3833c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.191129 18490 raft_consensus.cc:359] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.192879 18490 raft_consensus.cc:385] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.193179 18490 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.193930 18490 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.200862 18495 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.202618 18375 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:14.208477 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.209631 18500 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.216498 18500 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.221179 18290 tablet_service.cc:1511] Processing CreateTablet for tablet 0936d54d511a4cc3903264974a56ecb7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:14.222471 18290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0936d54d511a4cc3903264974a56ecb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.224545 18439 tablet_service.cc:1511] Processing CreateTablet for tablet 0936d54d511a4cc3903264974a56ecb7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:14.225821 18439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0936d54d511a4cc3903264974a56ecb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.227293 18495 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.227914 18495 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.228049 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:16:14.229425 18338 leader_election.cc:343] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.231772 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2ee63a449648bfb064fadee177cadb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:14.232013 18364 tablet_service.cc:1511] Processing CreateTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:14.233098 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2ee63a449648bfb064fadee177cadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.233280 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea246fa8cc7e4edbb136da2ee92fb1d3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:14.238540 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 110801us
I20260709 12:16:14.240012 18281 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0efe89255942c4bd576ff4b96cd91d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:14.240311 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.241211 18281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0efe89255942c4bd576ff4b96cd91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.248314 18449 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.249727 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:16:14.255357 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:14.259610 18365 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2ee63a449648bfb064fadee177cadb (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:14.260823 18365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2ee63a449648bfb064fadee177cadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.263375 18495 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.263847 18495 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.264696 18495 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.264885 18363 tablet_service.cc:1511] Processing CreateTablet for tablet 0936d54d511a4cc3903264974a56ecb7 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:14.266058 18363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0936d54d511a4cc3903264974a56ecb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.251708 18356 tablet_service.cc:1511] Processing CreateTablet for tablet e7d0369b4aea4afb960bf859e9b7738c (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:14.276868 18356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d0369b4aea4afb960bf859e9b7738c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.278151 18430 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0efe89255942c4bd576ff4b96cd91d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:14.279415 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0efe89255942c4bd576ff4b96cd91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.282162 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0efe89255942c4bd576ff4b96cd91d (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:14.283438 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0efe89255942c4bd576ff4b96cd91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.283385 18358 tablet_service.cc:1511] Processing CreateTablet for tablet 67a1783f49244bd5ba98721205d1ff6e (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:14.286470 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.287742 18358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a1783f49244bd5ba98721205d1ff6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:14.288067 18264 leader_election.cc:343] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.292595 18264 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.292587 18284 tablet_service.cc:1511] Processing CreateTablet for tablet 83652a40670f4224a56a5a1347b9d1a1 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:14.293851 18499 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.294142 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83652a40670f4224a56a5a1347b9d1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.300406 18491 tablet_bootstrap.cc:654] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.309165 18492 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.309755 18492 ts_tablet_manager.cc:1403] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.293s	user 0.030s	sys 0.047s
I20260709 12:16:14.311194 18490 ts_tablet_manager.cc:1434] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.123s	user 0.018s	sys 0.016s
I20260709 12:16:14.312286 18490 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.316023 18359 tablet_service.cc:1511] Processing CreateTablet for tablet 83652a40670f4224a56a5a1347b9d1a1 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:14.322420 18516 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.322921 18516 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.322955 18435 tablet_service.cc:1511] Processing CreateTablet for tablet 83652a40670f4224a56a5a1347b9d1a1 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:14.326232 18516 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.326159 18492 raft_consensus.cc:359] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.332942 18357 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd86332c91f4c8d8616b22b5daaf743 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:14.334313 18357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd86332c91f4c8d8616b22b5daaf743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.326992 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.327396 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.337270 18197 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:14.338680 18339 leader_election.cc:343] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.331277 18492 raft_consensus.cc:385] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.339370 18492 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.337357 18299 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.341220 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:14.341976 18285 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd86332c91f4c8d8616b22b5daaf743 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:14.342329 18516 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.342868 18492 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.343351 18285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd86332c91f4c8d8616b22b5daaf743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.346199 18435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83652a40670f4224a56a5a1347b9d1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.343875 18499 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.347576 18499 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.350455 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:14.351982 18266 leader_election.cc:343] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.352977 18499 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.354182 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:14.355320 18264 leader_election.cc:343] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.355746 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.357420 18492 ts_tablet_manager.cc:1434] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:16:14.357530 18499 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.343485 18516 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.359001 18516 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.359181 18284 tablet_service.cc:1511] Processing CreateTablet for tablet 8e62e9fad92b449d9315da4aa77082b4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:14.360462 18284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e62e9fad92b449d9315da4aa77082b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.360270 18438 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd86332c91f4c8d8616b22b5daaf743 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:14.361963 18492 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.366987 18359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83652a40670f4224a56a5a1347b9d1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.367496 18436 tablet_service.cc:1511] Processing CreateTablet for tablet 8e62e9fad92b449d9315da4aa77082b4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:14.372202 18438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd86332c91f4c8d8616b22b5daaf743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.376166 18361 tablet_service.cc:1511] Processing CreateTablet for tablet 8e62e9fad92b449d9315da4aa77082b4 (DEFAULT_TABLE table=test-workload [id=8347973bc73f4ec692d005859baada6b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:14.377354 18361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e62e9fad92b449d9315da4aa77082b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.364910 18490 tablet_bootstrap.cc:654] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.384016 18436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e62e9fad92b449d9315da4aa77082b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.388379 18516 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.392244 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.392827 18299 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.397827 18491 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.398425 18491 ts_tablet_manager.cc:1403] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.334s	user 0.004s	sys 0.009s
I20260709 12:16:14.407015 18491 raft_consensus.cc:359] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.407745 18491 raft_consensus.cc:385] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.408038 18491 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.419883 18491 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.421181 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.423061 18491 ts_tablet_manager.cc:1434] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 12:16:14.423677 18299 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:14.426541 18339 leader_election.cc:343] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.426919 18516 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.433920 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:14.434192 18491 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.434696 18516 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.435819 18516 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.437497 18516 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.443886 18490 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.444468 18490 ts_tablet_manager.cc:1403] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.132s	user 0.005s	sys 0.007s
I20260709 12:16:14.445815 18492 tablet_bootstrap.cc:654] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.452395 18499 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.452817 18499 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.453395 18490 raft_consensus.cc:359] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.454202 18490 raft_consensus.cc:385] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.454510 18499 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.454535 18490 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.455943 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.455570 18490 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.456161 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:14.457360 18266 leader_election.cc:343] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:14.457962 18264 leader_election.cc:343] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.458318 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.458395 18490 ts_tablet_manager.cc:1434] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:16:14.459393 18499 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.462527 18490 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.472427 18490 tablet_bootstrap.cc:654] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.473466 18492 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.473990 18492 ts_tablet_manager.cc:1403] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.001s
I20260709 12:16:14.477226 18492 raft_consensus.cc:359] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.476583 18197 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:14.478078 18492 raft_consensus.cc:385] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.478470 18492 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.479434 18491 tablet_bootstrap.cc:654] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.479424 18492 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.481967 18492 ts_tablet_manager.cc:1434] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:14.483397 18492 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.490360 18490 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.490793 18490 ts_tablet_manager.cc:1403] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.010s
I20260709 12:16:14.493611 18490 raft_consensus.cc:359] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.494354 18490 raft_consensus.cc:385] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.494654 18490 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.495421 18490 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.497061 18492 tablet_bootstrap.cc:654] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.500690 18490 ts_tablet_manager.cc:1434] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:16:14.501883 18490 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.503188 18491 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.503739 18491 ts_tablet_manager.cc:1403] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.014s
I20260709 12:16:14.506803 18491 raft_consensus.cc:359] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.507959 18491 raft_consensus.cc:385] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.508304 18491 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.509186 18491 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.510569 18490 tablet_bootstrap.cc:654] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.518406 18492 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.518810 18492 ts_tablet_manager.cc:1403] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.015s
I20260709 12:16:14.519574 18490 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.519627 18491 ts_tablet_manager.cc:1434] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 12:16:14.520198 18490 ts_tablet_manager.cc:1403] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 12:16:14.520876 18491 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.521231 18492 raft_consensus.cc:359] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.521950 18492 raft_consensus.cc:385] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.522215 18492 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.523059 18492 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.523263 18490 raft_consensus.cc:359] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.524060 18490 raft_consensus.cc:385] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.524408 18490 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.525292 18490 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.527833 18490 ts_tablet_manager.cc:1434] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:14.528234 18492 ts_tablet_manager.cc:1434] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:16:14.528494 18491 tablet_bootstrap.cc:654] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.529178 18492 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.529201 18490 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.536094 18492 tablet_bootstrap.cc:654] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.538762 18490 tablet_bootstrap.cc:654] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.540424 18491 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.540982 18491 ts_tablet_manager.cc:1403] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 12:16:14.545012 18491 raft_consensus.cc:359] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.546084 18491 raft_consensus.cc:385] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.546595 18491 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.547937 18491 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.554364 18492 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.554852 18492 ts_tablet_manager.cc:1403] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.007s
I20260709 12:16:14.555253 18491 ts_tablet_manager.cc:1434] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 12:16:14.556654 18491 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.557636 18492 raft_consensus.cc:359] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.558221 18492 raft_consensus.cc:385] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.558471 18492 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.559118 18492 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.561381 18492 ts_tablet_manager.cc:1434] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:16:14.563557 18490 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.564110 18492 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.564145 18490 ts_tablet_manager.cc:1403] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.008s
I20260709 12:16:14.566012 18491 tablet_bootstrap.cc:654] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.567629 18490 raft_consensus.cc:359] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.568473 18490 raft_consensus.cc:385] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.568857 18490 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.569701 18490 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.571471 18492 tablet_bootstrap.cc:654] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.575425 18490 ts_tablet_manager.cc:1434] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:16:14.576566 18490 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.585170 18491 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.585677 18491 ts_tablet_manager.cc:1403] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.009s
I20260709 12:16:14.586117 18490 tablet_bootstrap.cc:654] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.588918 18491 raft_consensus.cc:359] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.589769 18491 raft_consensus.cc:385] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.590126 18491 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.590849 18491 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.591404 18492 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.591921 18492 ts_tablet_manager.cc:1403] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.020s
I20260709 12:16:14.592990 18490 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.593400 18490 ts_tablet_manager.cc:1403] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.002s
I20260709 12:16:14.593425 18491 ts_tablet_manager.cc:1434] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:14.594597 18491 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.594647 18492 raft_consensus.cc:359] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.595484 18492 raft_consensus.cc:385] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.595770 18492 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.596009 18490 raft_consensus.cc:359] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.596642 18490 raft_consensus.cc:385] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.597018 18490 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.596598 18492 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.597882 18490 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.598613 18516 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.599037 18516 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.602404 18491 tablet_bootstrap.cc:654] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.605712 18490 ts_tablet_manager.cc:1434] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:16:14.606527 18490 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.610908 18492 ts_tablet_manager.cc:1434] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:14.611922 18490 tablet_bootstrap.cc:654] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.611922 18492 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.617203 18491 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.617300 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.617779 18491 ts_tablet_manager.cc:1403] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
W20260709 12:16:14.618896 18338 leader_election.cc:343] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.620211 18492 tablet_bootstrap.cc:654] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.620710 18491 raft_consensus.cc:359] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.621457 18491 raft_consensus.cc:385] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.621750 18491 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.622470 18491 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.624027 18495 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.624794 18491 ts_tablet_manager.cc:1434] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:14.624482 18495 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.625355 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.626026 18449 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.626170 18491 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.626614 18495 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.627424 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.628090 18299 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.628137 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.629371 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.629685 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.630028 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.630759 18517 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.631167 18517 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.631542 18517 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.631829 18495 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.632203 18495 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.632579 18495 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.634194 18491 tablet_bootstrap.cc:654] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.635339 18516 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.641147 18495 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.644269 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.644865 18299 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.628454 18490 tablet_bootstrap.cc:492] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.645743 18490 ts_tablet_manager.cc:1403] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.004s
I20260709 12:16:14.648533 18490 raft_consensus.cc:359] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.649245 18490 raft_consensus.cc:385] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.649533 18490 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.650349 18490 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.652779 18490 ts_tablet_manager.cc:1434] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:14.654577 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.655251 18449 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.667443 18516 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.667984 18516 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.670178 18516 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.673424 18517 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.670809 18516 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:14.673908 18500 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.666910 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.678395 18500 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.687672 18500 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.688508 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.689167 18300 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:14.689756 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.690442 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.690582 18375 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:14.692323 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.693148 18298 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.693454 18299 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.691636 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.682088 18495 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.696043 18500 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.696419 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.696911 18500 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.697273 18500 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.697305 18495 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.697844 18495 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.697576 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.698860 18541 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:14.698969 18338 leader_election.cc:343] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.699421 18541 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.700348 18541 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.698369 18495 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.706291 18500 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.708433 18490 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.714123 18491 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.714608 18491 ts_tablet_manager.cc:1403] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.089s	user 0.021s	sys 0.044s
I20260709 12:16:14.715472 18492 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.715863 18492 ts_tablet_manager.cc:1403] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.104s	user 0.048s	sys 0.026s
I20260709 12:16:14.717252 18491 raft_consensus.cc:359] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.717798 18491 raft_consensus.cc:385] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.717613 18492 raft_consensus.cc:359] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.718075 18491 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.718250 18492 raft_consensus.cc:385] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.718555 18492 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.718750 18491 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.719194 18492 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:14.722198 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 61422.1us
I20260709 12:16:14.686822 18517 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.724577 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.725348 18448 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.726130 18500 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.727334 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:14.728873 18491 ts_tablet_manager.cc:1434] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:16:14.732317 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.733232 18490 tablet_bootstrap.cc:654] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.733357 18449 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:14.734642 18517 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.735149 18517 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.738065 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.738303 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.738662 18300 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.739003 18299 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.741343 18491 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.741119 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.742653 18516 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.743002 18516 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.743471 18516 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.743412 18500 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.744526 18500 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.747506 18542 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.747927 18491 tablet_bootstrap.cc:654] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.748028 18542 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.752362 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:14.753815 18338 leader_election.cc:343] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.755478 18517 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.755940 18542 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.753482 18516 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.757875 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.759572 18299 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:14.761149 18516 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.761555 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.767464 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.768092 18374 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:14.769423 18415 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:14.763007 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.771661 18449 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.773135 18197 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:14.777384 18448 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.778115 18516 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.778592 18516 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.781169 18516 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.781494 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.782169 18298 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.782473 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.783069 18448 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:14.783423 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.783468 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.766206 18500 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.789847 18492 ts_tablet_manager.cc:1434] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.074s	user 0.031s	sys 0.028s
I20260709 12:16:14.770592 18299 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.792062 18300 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:14.793021 18447 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.793551 18492 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.793883 18339 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.794991 18541 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.795365 18541 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.795672 18541 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.781399 18495 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.802384 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.799094 18500 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.799978 18516 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.804860 18517 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.810175 18516 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.806733 18375 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:14.810719 18516 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.812312 18499 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.812750 18499 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.807384 18492 tablet_bootstrap.cc:654] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.807624 18491 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.813982 18491 ts_tablet_manager.cc:1403] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.012s
I20260709 12:16:14.808180 18541 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.808488 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.809181 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.809787 18495 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.809127 18500 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.817584 18499 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.818290 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.819456 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.819957 18500 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.820042 18446 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
W20260709 12:16:14.820295 18266 leader_election.cc:343] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.822062 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:14.822009 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.823587 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.824862 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:14.825342 18500 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.825489 18298 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:14.825857 18500 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:16:14.826833 18415 leader_election.cc:343] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.826714 18500 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.827402 18544 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.827517 18499 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.827872 18544 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.827991 18499 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.828256 18544 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.828392 18499 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.829495 18298 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.829411 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.824724 18491 raft_consensus.cc:359] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.830583 18491 raft_consensus.cc:385] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.830997 18491 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.832578 18491 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.813844 18447 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.835500 18541 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.818854 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:14.838502 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.842744 18297 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.857633 18517 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.858642 18517 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.851601 18300 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:14.849887 18544 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.868484 18516 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.857339 18299 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.872316 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:14.874230 18547 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:14.874763 18339 leader_election.cc:343] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.874979 18547 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.875337 18547 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.875859 18491 ts_tablet_manager.cc:1434] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.062s	user 0.015s	sys 0.009s
I20260709 12:16:14.874671 18551 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.876792 18491 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.884035 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.884318 18516 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.884625 18298 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.880409 18297 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.877631 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:14.879529 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.887683 18375 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:14.880925 18339 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:14.880741 18551 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.888619 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.889901 18491 tablet_bootstrap.cc:654] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.889622 18551 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:14.889647 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.887732 18300 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.896780 18516 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.897470 18516 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:16:14.897929 18339 leader_election.cc:343] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.895493 18499 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.898296 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.898274 18516 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.899289 18449 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.900434 18499 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:14.876734 18544 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.905294 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:14.909296 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:14.910415 18495 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:14.911315 18266 leader_election.cc:343] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.912539 18300 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:14.915688 18550 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260709 12:16:14.913619 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:14.917949 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:14.919448 18197 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:14.922397 18449 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.926151 18339 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:14.929409 18375 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:14.929881 18552 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.930382 18552 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:14.931931 18490 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:14.932458 18490 ts_tablet_manager.cc:1403] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.224s	user 0.014s	sys 0.002s
I20260709 12:16:14.935032 18298 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:14.937361 18490 raft_consensus.cc:359] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.937932 18490 raft_consensus.cc:385] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.938154 18490 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.940891 18491 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:14.941339 18491 ts_tablet_manager.cc:1403] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.023s
I20260709 12:16:14.943758 18491 raft_consensus.cc:359] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.931164 18552 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.944422 18491 raft_consensus.cc:385] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.944691 18491 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.945298 18491 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.947299 18491 ts_tablet_manager.cc:1434] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:14.947569 18490 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.948120 18491 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:14.951560 18553 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.952144 18553 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:14.954428 18490 ts_tablet_manager.cc:1434] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.001s
I20260709 12:16:14.955133 18490 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:14.958809 18490 tablet_bootstrap.cc:654] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.959762 18547 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.952982 18553 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.965083 18492 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:14.965492 18492 ts_tablet_manager.cc:1403] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.172s	user 0.009s	sys 0.008s
I20260709 12:16:14.967674 18492 raft_consensus.cc:359] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.968315 18492 raft_consensus.cc:385] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.968590 18492 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.969194 18492 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:14.971577 18492 ts_tablet_manager.cc:1434] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:14.972566 18492 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:14.973492 18446 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:14.975185 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:14.976404 18499 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.977907 18491 tablet_bootstrap.cc:654] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.979615 18198 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:14.984933 18547 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:14.995375 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:14.996186 18300 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:15.003618 18492 tablet_bootstrap.cc:654] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.006145 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.006757 18375 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.013789 18491 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.014247 18491 ts_tablet_manager.cc:1403] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.007s
I20260709 12:16:15.016806 18491 raft_consensus.cc:359] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.017431 18491 raft_consensus.cc:385] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.017699 18491 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.020565 18491 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.023432 18491 ts_tablet_manager.cc:1434] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:16:15.024106 18499 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.024323 18491 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.023705 18197 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.024573 18499 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.026571 18499 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.026911 18375 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.028635 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.031551 18490 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.032282 18490 ts_tablet_manager.cc:1403] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.011s
I20260709 12:16:15.032341 18415 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:15.033934 18553 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.034379 18553 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:15.035462 18491 tablet_bootstrap.cc:654] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.035140 18553 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.027685 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:15.041167 18266 leader_election.cc:343] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:15.041841 18264 leader_election.cc:343] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.042814 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.047137 18490 raft_consensus.cc:359] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.047823 18490 raft_consensus.cc:385] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.045418 18198 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.048092 18490 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.048051 18551 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.049034 18490 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.048852 18198 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.050523 18198 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.051211 18490 ts_tablet_manager.cc:1434] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:15.047623 18499 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.052647 18490 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.061744 18492 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.062184 18492 ts_tablet_manager.cc:1403] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.014s
I20260709 12:16:15.066745 18492 raft_consensus.cc:359] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.067240 18492 raft_consensus.cc:385] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.067543 18492 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.070977 18490 tablet_bootstrap.cc:654] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.068212 18492 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.078186 18492 ts_tablet_manager.cc:1434] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:16:15.079502 18492 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
W20260709 12:16:15.081202 18563 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:16:15.079386 18195 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.085265 18491 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.085716 18491 ts_tablet_manager.cc:1403] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.062s	user 0.025s	sys 0.025s
I20260709 12:16:15.086467 18490 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.086925 18490 ts_tablet_manager.cc:1403] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 12:16:15.088325 18491 raft_consensus.cc:359] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.088953 18491 raft_consensus.cc:385] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.089192 18491 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.089834 18491 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.089880 18490 raft_consensus.cc:359] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.090839 18490 raft_consensus.cc:385] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.092794 18491 ts_tablet_manager.cc:1434] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:16:15.093305 18517 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.093644 18491 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.093760 18517 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.095249 18492 tablet_bootstrap.cc:654] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.095858 18517 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.096946 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.097405 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:15.098344 18339 leader_election.cc:343] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:15.099203 18338 leader_election.cc:343] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.099747 18491 tablet_bootstrap.cc:654] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.099781 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:15.101222 18517 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.091111 18490 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.102862 18490 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.104987 18490 ts_tablet_manager.cc:1434] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:16:15.105866 18490 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.106050 18492 tablet_bootstrap.cc:492] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.106458 18492 ts_tablet_manager.cc:1403] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.008s
I20260709 12:16:15.111001 18490 tablet_bootstrap.cc:654] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.115648 18492 raft_consensus.cc:359] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.116369 18492 raft_consensus.cc:385] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.116580 18492 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.117380 18492 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.117960 18491 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.118388 18491 ts_tablet_manager.cc:1403] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:16:15.118944 18492 ts_tablet_manager.cc:1434] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:16:15.119833 18492 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:15.120752 18491 raft_consensus.cc:359] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.121165 18491 raft_consensus.cc:385] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.121327 18491 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.121810 18491 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.123283 18491 ts_tablet_manager.cc:1434] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:16:15.123879 18491 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.129638 18491 tablet_bootstrap.cc:654] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.135411 18492 tablet_bootstrap.cc:654] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.137265 18491 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.137560 18491 ts_tablet_manager.cc:1403] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:16:15.139564 18491 raft_consensus.cc:359] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.140205 18491 raft_consensus.cc:385] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.140383 18491 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.140980 18490 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.140884 18491 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.141269 18492 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.141531 18490 ts_tablet_manager.cc:1403] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.036s	user 0.027s	sys 0.001s
I20260709 12:16:15.142284 18491 ts_tablet_manager.cc:1434] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:16:15.142680 18492 ts_tablet_manager.cc:1403] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.004s
I20260709 12:16:15.143267 18491 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.144418 18490 raft_consensus.cc:359] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.145119 18490 raft_consensus.cc:385] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.145417 18490 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.145605 18492 raft_consensus.cc:359] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.146291 18492 raft_consensus.cc:385] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.146637 18492 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.146312 18490 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.147392 18492 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.148917 18490 ts_tablet_manager.cc:1434] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:15.149343 18491 tablet_bootstrap.cc:654] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.150357 18490 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.155850 18490 tablet_bootstrap.cc:654] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.156024 18492 ts_tablet_manager.cc:1434] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:15.157004 18492 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:15.160084 18491 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.160490 18491 ts_tablet_manager.cc:1403] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 12:16:15.162972 18491 raft_consensus.cc:359] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.163720 18491 raft_consensus.cc:385] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.164001 18491 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.164057 18492 tablet_bootstrap.cc:654] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.164824 18491 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.167026 18491 ts_tablet_manager.cc:1434] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:15.167874 18491 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.173334 18491 tablet_bootstrap.cc:654] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.174263 18490 tablet_bootstrap.cc:492] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.174752 18490 ts_tablet_manager.cc:1403] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.008s
I20260709 12:16:15.178210 18490 raft_consensus.cc:359] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.179307 18490 raft_consensus.cc:385] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.179749 18490 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.180776 18490 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.183848 18490 ts_tablet_manager.cc:1434] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:15.185063 18490 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.191161 18554 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.191671 18554 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.193447 18490 tablet_bootstrap.cc:654] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.194947 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.195355 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:15.196431 18264 leader_election.cc:343] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:15.196979 18266 leader_election.cc:343] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.196898 18517 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.197454 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.198290 18499 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.199080 18554 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.201162 18492 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.201710 18492 ts_tablet_manager.cc:1403] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.019s
I20260709 12:16:15.201932 18491 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.202721 18491 ts_tablet_manager.cc:1403] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.005s
I20260709 12:16:15.205168 18492 raft_consensus.cc:359] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.206125 18492 raft_consensus.cc:385] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.206135 18491 raft_consensus.cc:359] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.206730 18492 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.207068 18491 raft_consensus.cc:385] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.207415 18491 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.207937 18492 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.208169 18491 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.210536 18491 ts_tablet_manager.cc:1434] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:15.210726 18492 ts_tablet_manager.cc:1434] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:15.211474 18491 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.211912 18492 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:15.217972 18491 tablet_bootstrap.cc:654] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.230865 18551 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.220458 18492 tablet_bootstrap.cc:654] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.240777 18491 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.241384 18491 ts_tablet_manager.cc:1403] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.012s
I20260709 12:16:15.243916 18491 raft_consensus.cc:359] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.244565 18491 raft_consensus.cc:385] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.244835 18491 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.245741 18491 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.248386 18491 ts_tablet_manager.cc:1434] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:15.249229 18491 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.250864 18554 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.251370 18554 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.253559 18554 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.255338 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:15.257059 18266 leader_election.cc:343] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.258304 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:15.259701 18264 leader_election.cc:343] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.260149 18264 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.261168 18554 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.266374 18491 tablet_bootstrap.cc:654] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:15.267808 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:15.278105 18553 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.290251 18553 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.305752 18551 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.305966 18449 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.317862 18446 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.330121 18490 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.330641 18490 ts_tablet_manager.cc:1403] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.146s	user 0.050s	sys 0.056s
I20260709 12:16:15.335314 18550 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.335314 18551 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:15.341059 18492 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.341502 18492 ts_tablet_manager.cc:1403] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.130s	user 0.028s	sys 0.044s
I20260709 12:16:15.337692 18490 raft_consensus.cc:359] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.343257 18490 raft_consensus.cc:385] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.343619 18490 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.348250 18490 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.353914 18492 raft_consensus.cc:359] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.354828 18492 raft_consensus.cc:385] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.355216 18492 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.356209 18492 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.359148 18492 ts_tablet_manager.cc:1434] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 12:16:15.364012 18544 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.371289 18490 ts_tablet_manager.cc:1434] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 12:16:15.372844 18490 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.364017 18492 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:15.384702 18551 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.385946 18553 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:15.384783 18544 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.386859 18553 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.383440 18550 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:15.387614 18544 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.389536 18544 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.392501 18492 tablet_bootstrap.cc:654] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.393070 18550 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.395685 18491 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.396219 18491 ts_tablet_manager.cc:1403] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.147s	user 0.017s	sys 0.042s
I20260709 12:16:15.397100 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.398591 18490 tablet_bootstrap.cc:654] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.397727 18375 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.399987 18491 raft_consensus.cc:359] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.400782 18491 raft_consensus.cc:385] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.401149 18491 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.404147 18553 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.405798 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.406446 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.404904 18491 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.416240 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.419988 18553 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.420437 18553 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.420817 18553 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.421808 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.446557 18544 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.447274 18544 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.450044 18544 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.459055 18544 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.459636 18544 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.463866 18491 ts_tablet_manager.cc:1434] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.067s	user 0.013s	sys 0.021s
I20260709 12:16:15.464627 18491 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.464913 18554 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.465694 18554 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.466104 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.466384 18544 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.463668 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:15.470610 18415 leader_election.cc:343] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.471036 18554 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.472200 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.471705 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.472954 18375 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
W20260709 12:16:15.473517 18266 leader_election.cc:343] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.474465 18415 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.475572 18544 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.477499 18553 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.482321 18544 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.483541 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.484530 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.486843 18299 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.485028 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.482933 18300 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.487437 18544 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.495829 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:15.497047 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.489069 18500 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.489648 18296 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.503959 18500 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:15.504786 18500 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.505177 18500 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.505638 18500 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.508790 18499 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.509389 18499 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.507647 18554 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:15.508297 18264 leader_election.cc:343] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.518425 18553 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.520469 18499 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.526441 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.529029 18374 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.529466 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.517462 18554 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.535424 18554 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.530372 18373 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:15.536365 18492 tablet_bootstrap.cc:492] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.537163 18492 ts_tablet_manager.cc:1403] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.173s	user 0.011s	sys 0.009s
I20260709 12:16:15.532224 18544 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.538841 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:15.539908 18554 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.540266 18554 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.540673 18554 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.541504 18544 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.517813 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.538156 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.527951 18598 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.544225 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.546331 18598 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.550565 18499 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.547482 18373 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.565337 18490 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.565946 18490 ts_tablet_manager.cc:1403] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.193s	user 0.035s	sys 0.038s
W20260709 12:16:15.579808 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 77346.9us
I20260709 12:16:15.582990 18491 tablet_bootstrap.cc:654] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.584940 18490 raft_consensus.cc:359] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.584196 18492 raft_consensus.cc:359] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.585412 18374 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.585989 18492 raft_consensus.cc:385] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.585628 18490 raft_consensus.cc:385] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.587114 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.585929 18598 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.588665 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.588826 18375 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.589274 18446 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:15.588331 18490 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.587081 18492 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.590221 18500 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.591219 18490 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.581609 18552 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.591686 18492 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.594874 18499 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.606194 18490 ts_tablet_manager.cc:1434] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 12:16:15.607074 18490 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.607956 18492 ts_tablet_manager.cc:1434] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.070s	user 0.011s	sys 0.011s
I20260709 12:16:15.608400 18551 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.608855 18551 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.597662 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.612393 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.613170 18547 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.694255 18492 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
W20260709 12:16:15.597159 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 99668.6us
I20260709 12:16:15.725183 18551 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.607954 18499 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.624734 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.615978 18544 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.730604 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.693208 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.691769 18500 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.731956 18375 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:15.733870 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:16:15.724246 18601 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:16:15.616073 18547 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:15.690925 18598 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:15.684880 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69475.3us
I20260709 12:16:15.733814 18554 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.740538 18499 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:16:15.740669 18415 leader_election.cc:343] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.731459 18544 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.742252 18544 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.743185 18499 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.747720 18499 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.746640 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.748075 18499 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.743501 18547 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.742218 18598 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.758256 18554 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.758703 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:15.759305 18298 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.763767 18446 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:15.765020 18490 tablet_bootstrap.cc:654] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.766877 18598 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.760023 18373 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.768075 18544 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.772025 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.772854 18374 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:15.774592 18415 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.775074 18492 tablet_bootstrap.cc:654] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.776330 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.777678 18598 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.778154 18602 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.778534 18602 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.778903 18602 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.781575 18491 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:15.782001 18491 ts_tablet_manager.cc:1403] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.318s	user 0.009s	sys 0.015s
I20260709 12:16:15.784555 18544 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:15.784372 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:15.776340 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.785121 18299 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.785650 18491 raft_consensus.cc:359] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.786283 18491 raft_consensus.cc:385] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.786526 18491 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.786993 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.788275 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.789578 18374 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.782590 18598 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:15.791554 18598 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.782747 18264 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.791977 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.799671 18550 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.801345 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:15.802095 18448 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:15.803522 18544 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.800859 18299 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.794348 18499 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.808221 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.808646 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.812019 18374 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.822614 18300 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:15.823659 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.826582 18491 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.827590 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:15.828226 18299 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.814085 18297 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:15.859001 18490 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:15.859687 18490 ts_tablet_manager.cc:1403] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.253s	user 0.012s	sys 0.007s
I20260709 12:16:15.873346 18603 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.875211 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.875813 18374 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.876765 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:15.867683 18499 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.882215 18491 ts_tablet_manager.cc:1434] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.100s	user 0.026s	sys 0.025s
I20260709 12:16:15.883085 18491 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:15.884966 18554 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:15.870010 18602 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.879709 18547 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.884873 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.888474 18608 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.889453 18608 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.888513 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:15.888068 18547 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:15.891018 18547 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.891710 18490 raft_consensus.cc:359] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.892342 18490 raft_consensus.cc:385] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.892606 18490 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.892686 18608 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.893244 18490 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.901229 18490 ts_tablet_manager.cc:1434] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:16:15.903180 18490 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:15.904368 18449 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:15.905898 18299 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:15.903678 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:15.907766 18294 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:15.907920 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.909267 18551 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.909715 18551 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.911865 18602 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:16:15.915454 18264 leader_election.cc:343] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.915477 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:15.916538 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:15.920035 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.935828 18517 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.936421 18517 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.939751 18551 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.915167 18602 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:16:15.954561 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:15.954841 18517 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.954986 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:15.955380 18294 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:15.956454 18300 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
W20260709 12:16:15.957877 18266 leader_election.cc:343] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.957988 18554 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.957183 18374 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:15.958300 18266 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.958642 18552 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.959304 18492 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:15.959625 18613 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.957878 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:15.958503 18554 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.960301 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:15.961463 18613 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.961910 18613 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:15.962101 18554 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.962772 18613 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.963841 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:15.966502 18264 leader_election.cc:343] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.964148 18517 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.966866 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:15.967368 18517 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.967715 18517 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:15.968170 18266 leader_election.cc:343] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.968632 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:15.966511 18491 tablet_bootstrap.cc:654] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.970023 18492 ts_tablet_manager.cc:1403] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.276s	user 0.018s	sys 0.006s
I20260709 12:16:15.970526 18608 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.971943 18195 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.973726 18490 tablet_bootstrap.cc:654] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.974015 18492 raft_consensus.cc:359] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.974658 18492 raft_consensus.cc:385] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.974900 18492 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.975481 18492 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.981294 18492 ts_tablet_manager.cc:1434] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:16:15.984082 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.988229 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:15.991539 18517 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:15.992022 18492 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:15.985854 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.003643 18517 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:15.996938 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:16.015954 18295 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.035315 18492 tablet_bootstrap.cc:654] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.009270 18299 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:16.052294 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.053468 18517 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.053910 18517 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.054363 18517 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.025573 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:16.060355 18598 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:16.060526 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51018.3us
I20260709 12:16:16.061031 18598 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:16.062017 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.062175 18598 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.065506 18547 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:16.066959 18339 leader_election.cc:343] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:16.071841 18339 leader_election.cc:343] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.081670 18490 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.082271 18490 ts_tablet_manager.cc:1403] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.179s	user 0.009s	sys 0.005s
I20260709 12:16:16.085702 18490 raft_consensus.cc:359] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.086618 18490 raft_consensus.cc:385] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.086963 18490 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.088508 18490 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.090734 18490 ts_tablet_manager.cc:1434] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:16.095170 18491 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.095877 18491 ts_tablet_manager.cc:1403] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.213s	user 0.045s	sys 0.037s
W20260709 12:16:16.095825 18617 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:16:16.098968 18491 raft_consensus.cc:359] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.100106 18492 tablet_bootstrap.cc:492] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.100589 18492 ts_tablet_manager.cc:1403] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.109s	user 0.003s	sys 0.009s
I20260709 12:16:16.100598 18490 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.101473 18517 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.103634 18295 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.103045 18492 raft_consensus.cc:359] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.104122 18492 raft_consensus.cc:385] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.104378 18492 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.104933 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.105103 18492 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.105677 18552 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.106082 18552 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:16.107149 18492 ts_tablet_manager.cc:1434] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:16:16.099995 18491 raft_consensus.cc:385] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.107869 18491 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.108319 18492 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.106869 18552 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.113359 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.116328 18491 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.118516 18491 ts_tablet_manager.cc:1434] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:16:16.119414 18491 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
W20260709 12:16:16.119869 18339 leader_election.cc:343] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.126214 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:16.126876 18295 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.129854 18491 tablet_bootstrap.cc:654] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.132113 18517 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.134533 18492 tablet_bootstrap.cc:654] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.138340 18517 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.138823 18517 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.139268 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.139866 18449 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.142302 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.127780 18196 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.143832 18296 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:16.143848 18197 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.149132 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.150554 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.151325 18490 tablet_bootstrap.cc:654] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.151435 18552 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.141783 18517 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.162384 18552 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.153815 18449 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.166373 18492 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.166807 18492 ts_tablet_manager.cc:1403] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.013s
I20260709 12:16:16.179625 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:16.180678 18517 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.181244 18517 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:16.181993 18517 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.183789 18598 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.187662 18491 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.188102 18491 ts_tablet_manager.cc:1403] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.016s
I20260709 12:16:16.187875 18492 raft_consensus.cc:359] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.188457 18492 raft_consensus.cc:385] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.188738 18492 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.189324 18492 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.192799 18491 raft_consensus.cc:359] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.193722 18491 raft_consensus.cc:385] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.194098 18491 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.194325 18492 ts_tablet_manager.cc:1434] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 12:16:16.196877 18492 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.199747 18196 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.206146 18491 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.208063 18491 ts_tablet_manager.cc:1434] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:16:16.208972 18491 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.209290 18492 tablet_bootstrap.cc:654] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.210100 18196 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.211982 18196 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.214576 18491 tablet_bootstrap.cc:654] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.206161 18295 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.215428 18547 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.215967 18547 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.218055 18490 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.218006 18547 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:16.218873 18490 ts_tablet_manager.cc:1403] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.004s
I20260709 12:16:16.219658 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.220446 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.215167 18492 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.221434 18492 ts_tablet_manager.cc:1403] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.002s
I20260709 12:16:16.221841 18490 raft_consensus.cc:359] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:16.222265 18415 leader_election.cc:343] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.222515 18490 raft_consensus.cc:385] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.222977 18490 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.222870 18547 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.223804 18490 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.224504 18492 raft_consensus.cc:359] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.225165 18492 raft_consensus.cc:385] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.225435 18492 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
W20260709 12:16:16.226100 18412 leader_election.cc:343] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.226053 18492 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.226783 18490 ts_tablet_manager.cc:1434] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:16.226616 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.227792 18490 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.227769 18544 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.228056 18492 ts_tablet_manager.cc:1434] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:16.228993 18492 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.229518 18608 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.230002 18608 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.233587 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.234354 18374 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:16.233683 18608 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.235319 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.235617 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.236208 18449 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:16.238304 18608 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.238932 18608 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.239598 18608 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.240692 18490 tablet_bootstrap.cc:654] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.249001 18195 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.253572 18608 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.255625 18492 tablet_bootstrap.cc:654] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.257709 18491 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.258309 18491 ts_tablet_manager.cc:1403] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.007s
I20260709 12:16:16.255692 18608 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.260048 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:16.261559 18375 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.260542 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.263532 18449 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.263585 18491 raft_consensus.cc:359] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.264154 18491 raft_consensus.cc:385] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.264323 18491 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.264882 18491 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.266753 18491 ts_tablet_manager.cc:1434] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:16:16.267639 18491 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.275074 18547 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.285931 18375 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:16.289479 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.290386 18608 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.290534 18490 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.290990 18490 ts_tablet_manager.cc:1403] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.006s
I20260709 12:16:16.291034 18608 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:16.291307 18547 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:16.285085 18449 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:16.293816 18608 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.294727 18490 raft_consensus.cc:359] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.295630 18490 raft_consensus.cc:385] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.295902 18490 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.296516 18490 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.298992 18490 ts_tablet_manager.cc:1434] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:16:16.301869 18492 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.302312 18492 ts_tablet_manager.cc:1403] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.003s
I20260709 12:16:16.304823 18492 raft_consensus.cc:359] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.306007 18492 raft_consensus.cc:385] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.306293 18492 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.300275 18490 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.309042 18491 tablet_bootstrap.cc:654] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.311605 18492 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.313970 18492 ts_tablet_manager.cc:1434] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:16.314844 18490 tablet_bootstrap.cc:654] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.315371 18492 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.322346 18492 tablet_bootstrap.cc:654] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.328742 18517 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.329453 18491 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.329280 18517 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.329870 18491 ts_tablet_manager.cc:1403] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.000s
I20260709 12:16:16.331401 18517 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.332065 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.332432 18491 raft_consensus.cc:359] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.332662 18295 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:16.333880 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.335009 18491 raft_consensus.cc:385] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.335318 18491 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.335950 18491 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.337657 18491 ts_tablet_manager.cc:1434] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:16.338495 18491 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.344650 18491 tablet_bootstrap.cc:654] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.345424 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.347373 18517 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:16.347532 18339 leader_election.cc:343] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.348001 18517 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.348609 18517 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.342331 18198 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.362418 18490 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.362923 18490 ts_tablet_manager.cc:1403] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.004s
I20260709 12:16:16.363065 18517 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.365641 18490 raft_consensus.cc:359] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.365937 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:16.366350 18490 raft_consensus.cc:385] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.366525 18295 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.366634 18490 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.367332 18490 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.369379 18490 ts_tablet_manager.cc:1434] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:16.374522 18490 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.375640 18295 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.376850 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.377817 18616 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.378261 18616 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:16.379067 18616 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.380247 18490 tablet_bootstrap.cc:654] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.385762 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.387475 18550 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.388078 18550 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:16.388959 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:16:16.391870 18339 leader_election.cc:343] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.391959 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.391889 18492 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.392773 18492 ts_tablet_manager.cc:1403] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.078s	user 0.021s	sys 0.027s
I20260709 12:16:16.392675 18299 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:16.393570 18491 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.394013 18491 ts_tablet_manager.cc:1403] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.015s
I20260709 12:16:16.393922 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.394606 18448 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:16.394550 18517 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.396070 18492 raft_consensus.cc:359] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.396926 18492 raft_consensus.cc:385] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.396667 18491 raft_consensus.cc:359] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.397294 18492 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.397356 18491 raft_consensus.cc:385] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.397785 18491 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.398524 18491 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.398761 18492 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.399715 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.400787 18492 ts_tablet_manager.cc:1434] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:16.400928 18491 ts_tablet_manager.cc:1434] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:16.401796 18492 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.390864 18550 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.402135 18491 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.402308 18550 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.406457 18550 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.409300 18550 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.410501 18491 tablet_bootstrap.cc:654] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.414126 18492 tablet_bootstrap.cc:654] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.415132 18544 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:16.416864 18197 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.423465 18544 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.425510 18490 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.425958 18490 ts_tablet_manager.cc:1403] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.004s
I20260709 12:16:16.428807 18490 raft_consensus.cc:359] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.429445 18490 raft_consensus.cc:385] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.429725 18490 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.431023 18490 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.434125 18490 ts_tablet_manager.cc:1434] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:16.434890 18490 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.441175 18550 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.446373 18492 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.447181 18490 tablet_bootstrap.cc:654] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.449080 18492 ts_tablet_manager.cc:1403] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.019s
I20260709 12:16:16.451032 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.451826 18448 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.451076 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:16.452776 18295 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.460707 18448 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.448477 18550 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.462004 18608 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.464083 18608 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.465667 18492 raft_consensus.cc:359] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.467975 18492 raft_consensus.cc:385] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.468297 18492 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.469177 18491 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.469013 18492 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.469126 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:16.469749 18491 ts_tablet_manager.cc:1403] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.006s
I20260709 12:16:16.466255 18608 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.470186 18550 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.470781 18550 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:16.471565 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.480072 18295 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:16.482371 18264 leader_election.cc:343] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.471611 18550 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.472230 18491 raft_consensus.cc:359] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.483321 18491 raft_consensus.cc:385] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.483595 18491 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.484231 18491 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.486176 18491 ts_tablet_manager.cc:1434] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:16.473291 18544 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.487043 18491 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.494063 18491 tablet_bootstrap.cc:654] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.473855 18492 ts_tablet_manager.cc:1434] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.003s
I20260709 12:16:16.471056 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.496143 18492 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
W20260709 12:16:16.496496 18266 leader_election.cc:343] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.496562 18544 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.497102 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:16.497195 18544 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.498248 18608 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.499500 18490 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.499920 18490 ts_tablet_manager.cc:1403] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.012s
I20260709 12:16:16.503255 18490 raft_consensus.cc:359] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.503904 18490 raft_consensus.cc:385] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.504173 18490 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.504959 18616 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.504853 18490 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.505666 18492 tablet_bootstrap.cc:654] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.505481 18616 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.506875 18490 ts_tablet_manager.cc:1434] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:16.507797 18490 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:16.510063 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.510509 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.511339 18295 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:16:16.512439 18339 leader_election.cc:343] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.512451 18616 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.513302 18490 tablet_bootstrap.cc:654] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.517622 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.518417 18299 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:16.528026 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:16.529091 18517 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.529557 18517 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.529973 18517 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.534168 18608 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:16.540576 18517 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.543944 18608 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.544800 18608 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.547482 18608 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.551301 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:16.552114 18295 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.556258 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.557101 18449 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:16.558869 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.543895 18544 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:16.560173 18608 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.560698 18608 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.556281 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.561199 18608 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.561707 18375 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:16.577008 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:16:16.579160 18339 leader_election.cc:343] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.574079 18517 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.586437 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.665127 18613 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.700310 18613 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.701260 18550 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.711539 18550 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.721750 18491 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:16.722177 18491 ts_tablet_manager.cc:1403] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.235s	user 0.023s	sys 0.026s
I20260709 12:16:16.727195 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.729523 18446 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:16.730825 18490 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:16.731387 18490 ts_tablet_manager.cc:1403] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.224s	user 0.022s	sys 0.026s
I20260709 12:16:16.731544 18492 tablet_bootstrap.cc:492] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:16.731981 18492 ts_tablet_manager.cc:1403] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.236s	user 0.038s	sys 0.046s
I20260709 12:16:16.724262 18613 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.741164 18490 raft_consensus.cc:359] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.741853 18490 raft_consensus.cc:385] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.742066 18490 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.742573 18490 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.744966 18490 ts_tablet_manager.cc:1434] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:16:16.736711 18613 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.800860 18491 raft_consensus.cc:359] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.785902 18195 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.802287 18491 raft_consensus.cc:385] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.802580 18491 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.769785 18660 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
I20260709 12:16:16.774086 18550 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.805728 18490 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
W20260709 12:16:16.737062 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45849.3us
I20260709 12:16:16.748708 18492 raft_consensus.cc:359] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.807664 18492 raft_consensus.cc:385] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:16.803198 18491 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.807998 18492 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.799706 18415 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.808748 18492 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.809636 18602 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.810063 18491 ts_tablet_manager.cc:1434] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.088s	user 0.002s	sys 0.007s
I20260709 12:16:16.810750 18492 ts_tablet_manager.cc:1434] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.010s
I20260709 12:16:16.811725 18492 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:16.740119 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.796633 18658 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.818989 18491 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:16.810657 18608 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.804742 18613 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.826599 18608 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.819629 18658 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:16.820890 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 84172.8us
I20260709 12:16:16.831313 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.824402 18660 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:16.845571 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:16.848613 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.849768 18602 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.850917 18658 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:16.830945 18295 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:16.859334 18444 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.860921 18613 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.856156 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.857425 18660 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.858304 18374 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
W20260709 12:16:16.877290 18339 leader_election.cc:343] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.871730 18660 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.882277 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:16.882974 18415 leader_election.cc:343] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.885247 18662 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.878391 18616 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:16:16.887262 18444 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:16.888638 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:16.895926 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:16.896597 18373 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:16.898093 18608 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.898567 18608 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.896800 18490 tablet_bootstrap.cc:654] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.897625 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:16.900449 18608 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.913822 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:16.915589 18266 leader_election.cc:343] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.914381 18544 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:16:16.916308 18266 leader_election.cc:343] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.925340 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:16.927029 18613 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.927995 18613 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:16.929589 18613 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.922395 18657 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.940306 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:16.941332 18616 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.945120 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:16.946622 18616 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:16.947124 18616 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:16:16.947563 18412 leader_election.cc:343] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.947679 18373 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:16.948040 18412 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.948036 18616 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:16.948956 18544 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.951822 18491 tablet_bootstrap.cc:654] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:16.942454 18492 tablet_bootstrap.cc:654] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:16.954066 18339 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.954437 18338 leader_election.cc:343] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.954870 18338 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:16.968607 18196 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.959394 18552 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:16.984365 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:16.984534 18264 leader_election.cc:343] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:16.985083 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:16.987370 18661 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:16.988058 18552 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:16.986946 18198 catalog_manager.cc:5697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.002436 18670 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.013294 18660 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.015960 18660 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:16.998009 18667 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:16:17.018427 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:17.019683 18492 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.020268 18492 ts_tablet_manager.cc:1403] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.209s	user 0.014s	sys 0.026s
I20260709 12:16:17.018998 18374 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.031217 18490 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.031708 18490 ts_tablet_manager.cc:1403] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.230s	user 0.006s	sys 0.009s
I20260709 12:16:17.037629 18490 raft_consensus.cc:359] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.038302 18490 raft_consensus.cc:385] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.038563 18490 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.039206 18490 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.044690 18374 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
W20260709 12:16:17.028034 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43546.7us
I20260709 12:16:17.050653 18490 ts_tablet_manager.cc:1434] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:16:17.051602 18490 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.052910 18492 raft_consensus.cc:359] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.053551 18492 raft_consensus.cc:385] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.055824 18266 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.056105 18550 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:17.057926 18673 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.058761 18673 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:17.059667 18673 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.055009 18492 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.073487 18491 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.073251 18492 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.073953 18491 ts_tablet_manager.cc:1403] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.255s	user 0.020s	sys 0.025s
I20260709 12:16:17.076103 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.076738 18449 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.088032 18491 raft_consensus.cc:359] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.088723 18491 raft_consensus.cc:385] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.089933 18492 ts_tablet_manager.cc:1434] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:16:17.090858 18492 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.088991 18491 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.098075 18492 tablet_bootstrap.cc:654] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.097750 18491 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.098425 18550 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:17.103817 18264 leader_election.cc:343] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.104239 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.105372 18492 tablet_bootstrap.cc:492] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.105583 18449 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:17.105854 18492 ts_tablet_manager.cc:1403] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 12:16:17.107725 18657 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.111155 18491 ts_tablet_manager.cc:1434] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:16:17.110878 18492 raft_consensus.cc:359] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.111601 18492 raft_consensus.cc:385] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.111881 18492 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.112149 18491 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.112591 18492 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.116150 18492 ts_tablet_manager.cc:1434] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:17.117038 18492 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.116972 18195 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.118698 18490 tablet_bootstrap.cc:654] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.125962 18658 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.126387 18658 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.128402 18492 tablet_bootstrap.cc:654] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.128372 18658 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.133105 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.133123 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.134372 18300 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:17.137053 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.138536 18658 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.138952 18544 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.139415 18544 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.145120 18544 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.146595 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.146945 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:17.148480 18412 leader_election.cc:343] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.149034 18415 leader_election.cc:343] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.149622 18415 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.149644 18658 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.152470 18658 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.156829 18491 tablet_bootstrap.cc:654] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.160537 18492 tablet_bootstrap.cc:492] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.160997 18492 ts_tablet_manager.cc:1403] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.005s
I20260709 12:16:17.164008 18492 raft_consensus.cc:359] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.164675 18492 raft_consensus.cc:385] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.164984 18492 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.165969 18492 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.167309 18490 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.167732 18490 ts_tablet_manager.cc:1403] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.116s	user 0.024s	sys 0.010s
I20260709 12:16:17.168738 18492 ts_tablet_manager.cc:1434] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:16:17.169618 18492 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.170365 18490 raft_consensus.cc:359] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.171028 18490 raft_consensus.cc:385] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.171312 18490 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.171988 18490 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.173874 18490 ts_tablet_manager.cc:1434] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:17.175107 18490 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.175180 18492 tablet_bootstrap.cc:654] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.186280 18492 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.186722 18492 ts_tablet_manager.cc:1403] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 12:16:17.189270 18492 raft_consensus.cc:359] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.190735 18490 tablet_bootstrap.cc:654] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.191908 18492 raft_consensus.cc:385] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.192507 18492 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.195132 18492 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.197153 18492 ts_tablet_manager.cc:1434] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:17.198127 18492 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.203665 18492 tablet_bootstrap.cc:654] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.205237 18491 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.205835 18491 ts_tablet_manager.cc:1403] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.004s
I20260709 12:16:17.208371 18491 raft_consensus.cc:359] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.209071 18491 raft_consensus.cc:385] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.209339 18491 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.209911 18491 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.212296 18491 ts_tablet_manager.cc:1434] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:17.213174 18491 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.214744 18492 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.217049 18490 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.215168 18492 ts_tablet_manager.cc:1403] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 12:16:17.217486 18490 ts_tablet_manager.cc:1403] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.008s
I20260709 12:16:17.219076 18491 tablet_bootstrap.cc:654] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.219837 18492 raft_consensus.cc:359] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.220503 18492 raft_consensus.cc:385] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.220284 18490 raft_consensus.cc:359] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.220824 18492 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.220963 18490 raft_consensus.cc:385] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.221273 18490 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.222518 18490 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.225097 18490 ts_tablet_manager.cc:1434] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:17.226723 18490 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.230085 18491 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.230567 18491 ts_tablet_manager.cc:1403] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 12:16:17.221511 18492 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.232846 18492 ts_tablet_manager.cc:1434] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 12:16:17.233413 18491 raft_consensus.cc:359] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.234021 18491 raft_consensus.cc:385] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.234148 18492 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.234306 18491 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.235019 18491 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.237246 18491 ts_tablet_manager.cc:1434] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:17.238161 18491 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.240996 18492 tablet_bootstrap.cc:654] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.248299 18491 tablet_bootstrap.cc:654] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.243250 18490 tablet_bootstrap.cc:654] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.253556 18492 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.254027 18492 ts_tablet_manager.cc:1403] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 12:16:17.256162 18492 raft_consensus.cc:359] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.256783 18492 raft_consensus.cc:385] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.257050 18492 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.257212 18544 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.257845 18490 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.257682 18544 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.257694 18492 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.258342 18490 ts_tablet_manager.cc:1403] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:16:17.259852 18544 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.259948 18492 ts_tablet_manager.cc:1434] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:17.260910 18492 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.261632 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.262013 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.262238 18300 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:17.262612 18375 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:17.263558 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.265012 18544 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.265410 18544 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.265779 18544 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.267406 18490 raft_consensus.cc:359] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.268154 18490 raft_consensus.cc:385] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.268450 18490 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.269137 18490 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.271540 18491 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.271467 18490 ts_tablet_manager.cc:1434] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:16:17.272187 18491 ts_tablet_manager.cc:1403] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.005s
I20260709 12:16:17.273221 18490 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.272984 18544 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.275377 18491 raft_consensus.cc:359] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.276010 18491 raft_consensus.cc:385] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.275852 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:17.276338 18491 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.276556 18300 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.276978 18491 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.278926 18491 ts_tablet_manager.cc:1434] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:16:17.279753 18491 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.283828 18300 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:17.285308 18491 tablet_bootstrap.cc:654] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.285104 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.271431 18492 tablet_bootstrap.cc:654] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.288100 18658 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.288817 18658 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:17.290009 18658 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.304371 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:17.305162 18375 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.312681 18490 tablet_bootstrap.cc:654] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.321619 18375 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:17.323963 18490 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.365473 18490 ts_tablet_manager.cc:1403] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.000s
I20260709 12:16:17.368279 18490 raft_consensus.cc:359] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.368950 18490 raft_consensus.cc:385] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.369226 18490 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.332314 18491 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.369901 18491 ts_tablet_manager.cc:1403] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.090s	user 0.027s	sys 0.023s
I20260709 12:16:17.352475 18616 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:17.369838 18490 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.361408 18544 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.370898 18492 tablet_bootstrap.cc:492] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.340818 18613 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:17.371459 18492 ts_tablet_manager.cc:1403] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.111s	user 0.034s	sys 0.046s
I20260709 12:16:17.373723 18491 raft_consensus.cc:359] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.373925 18492 raft_consensus.cc:359] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.374351 18491 raft_consensus.cc:385] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.374508 18492 raft_consensus.cc:385] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.374642 18491 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.374800 18492 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.375852 18491 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.376715 18490 ts_tablet_manager.cc:1434] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:17.377841 18491 ts_tablet_manager.cc:1434] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 12:16:17.378295 18490 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.378698 18491 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.386548 18197 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:17.388583 18491 tablet_bootstrap.cc:654] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.395857 18447 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.397439 18492 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.399571 18492 ts_tablet_manager.cc:1434] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.002s
I20260709 12:16:17.399816 18491 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.399930 18490 tablet_bootstrap.cc:654] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.400208 18491 ts_tablet_manager.cc:1403] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 12:16:17.400417 18492 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.403561 18491 raft_consensus.cc:359] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.404290 18491 raft_consensus.cc:385] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.404556 18491 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.405471 18663 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.405938 18663 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.407328 18492 tablet_bootstrap.cc:654] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.408071 18663 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.410815 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.411691 18296 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:17.413455 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.415294 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.416102 18449 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:17.418321 18663 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.418886 18663 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.419778 18663 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.429590 18663 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.431133 18491 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.449468 18658 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.461647 18658 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.463409 18544 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.463759 18658 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.455514 18667 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.463884 18544 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.464727 18667 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.466770 18667 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.457206 18657 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:17.460348 18616 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:17.467180 18544 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.467211 18667 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.469758 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.470367 18667 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.470932 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.473194 18667 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.473563 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.468109 18658 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.474205 18294 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:17.474421 18658 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.475605 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.476373 18296 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
W20260709 12:16:17.476583 18415 leader_election.cc:343] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.477452 18415 leader_election.cc:343] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.468003 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.478790 18375 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:17.478999 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.482138 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.482156 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.483583 18444 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:17.484229 18658 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.484939 18658 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.485396 18658 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.485435 18412 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.485948 18658 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.487342 18657 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.487435 18544 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.487923 18544 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.487804 18657 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.488348 18544 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.488386 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.489928 18657 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:16:17.490178 18415 leader_election.cc:343] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.463840 18491 ts_tablet_manager.cc:1434] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:16:17.491557 18491 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.491819 18264 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.492558 18657 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.492890 18657 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.492951 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.493431 18657 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.495666 18613 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.495916 18673 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:17.496179 18613 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:17.498042 18264 leader_election.cc:343] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.498210 18613 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.500243 18657 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.500273 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:17.501876 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.502385 18296 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.503660 18657 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.503945 18266 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.505530 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:17.506785 18266 leader_election.cc:343] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.507354 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.508780 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.509881 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:16:17.511075 18264 leader_election.cc:343] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.514685 18670 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.515090 18670 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.515424 18670 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:17.516613 18412 leader_election.cc:343] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.517669 18616 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.518790 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.519026 18616 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.521896 18663 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.524242 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.524653 18616 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.524838 18446 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:17.518822 18412 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.525724 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.526522 18491 tablet_bootstrap.cc:654] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:17.516808 18266 leader_election.cc:343] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.527172 18338 leader_election.cc:343] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.528385 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.532840 18658 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.533972 18445 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:17.533967 18659 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.535159 18339 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.531488 18266 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.537307 18663 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.537689 18663 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.538064 18663 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.542378 18670 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.535925 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.537818 18544 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:17.548221 18266 leader_election.cc:343] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.551787 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.551877 18670 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.552474 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.551961 18492 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.553043 18445 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.553365 18296 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:17.553170 18492 ts_tablet_manager.cc:1403] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.153s	user 0.034s	sys 0.061s
I20260709 12:16:17.543677 18657 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.560832 18492 raft_consensus.cc:359] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.587893 18492 raft_consensus.cc:385] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.588407 18492 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.557037 18670 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.589519 18492 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:17.575922 18266 leader_election.cc:343] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.573379 18658 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.551257 18446 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.611227 18448 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:17.558262 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:16:17.611836 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53663.7us
I20260709 12:16:17.611915 18661 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.612494 18661 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:17.566474 18663 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.614892 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:17.618135 18491 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.618551 18491 ts_tablet_manager.cc:1403] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.127s	user 0.014s	sys 0.000s
W20260709 12:16:17.618999 18415 leader_election.cc:343] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.619573 18663 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.620370 18264 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.621151 18657 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.613981 18661 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.621973 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:16:17.623481 18338 leader_election.cc:343] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.588537 18445 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:17.629400 18492 ts_tablet_manager.cc:1434] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.075s	user 0.020s	sys 0.012s
I20260709 12:16:17.630331 18492 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.629433 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:17.632369 18300 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.632411 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.633143 18448 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.633222 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:17.634258 18373 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.636551 18670 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:17.640103 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:17.641245 18613 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.641686 18613 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:17.642469 18613 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.644403 18544 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:17.646668 18491 raft_consensus.cc:359] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.647356 18491 raft_consensus.cc:385] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.647629 18491 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.648239 18491 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.659860 18491 ts_tablet_manager.cc:1434] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:16:17.660836 18491 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.659265 18448 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:17.665108 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:17.669144 18446 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:17.670831 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:17.671607 18296 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:17.673648 18339 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.675191 18616 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.675351 18300 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:17.675704 18616 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:17.676545 18616 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.681412 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.682407 18658 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.685462 18663 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.686709 18490 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.687067 18490 ts_tablet_manager.cc:1403] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.310s	user 0.041s	sys 0.070s
I20260709 12:16:17.685920 18663 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.683032 18658 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:17.661469 18373 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
W20260709 12:16:17.685945 18415 leader_election.cc:343] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.698333 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:17.705948 18490 raft_consensus.cc:359] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.706658 18490 raft_consensus.cc:385] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.706909 18490 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.712693 18490 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.715984 18490 ts_tablet_manager.cc:1434] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.001s
I20260709 12:16:17.697798 18415 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.717674 18544 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.718568 18658 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.720697 18490 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.721705 18492 tablet_bootstrap.cc:654] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.721563 18195 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.729082 18663 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.731899 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.732957 18447 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:17.740365 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:17.749635 18616 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:17.750257 18616 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.750625 18616 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.761786 18197 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.798249 18544 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:17.830582 18658 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:17.839555 18661 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.840233 18661 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.842795 18661 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.852003 18490 tablet_bootstrap.cc:654] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.858357 18491 tablet_bootstrap.cc:654] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.861511 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.864472 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.865744 18295 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:17.864356 18616 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.867627 18616 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.868160 18613 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:17.868608 18613 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.870359 18195 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:17.879122 18339 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.880942 18613 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:17.884284 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:17.883993 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:17.885164 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:17.885305 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:17.885870 18447 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:17.886544 18266 leader_election.cc:343] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.887931 18338 leader_election.cc:343] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:17.887984 18264 leader_election.cc:343] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.888510 18264 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:17.888777 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:17.895032 18661 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:17.896533 18613 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.901536 18447 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:17.907353 18338 leader_election.cc:343] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:17.909343 18492 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.909742 18492 ts_tablet_manager.cc:1403] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.280s	user 0.010s	sys 0.014s
I20260709 12:16:17.910312 18491 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.910677 18491 ts_tablet_manager.cc:1403] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.250s	user 0.010s	sys 0.007s
I20260709 12:16:17.912565 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:17.913367 18550 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:17.913842 18550 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:17.914618 18550 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.917878 18490 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.918321 18490 ts_tablet_manager.cc:1403] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.198s	user 0.009s	sys 0.002s
I20260709 12:16:17.919107 18661 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:17.936591 18491 raft_consensus.cc:359] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.937265 18491 raft_consensus.cc:385] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.937556 18491 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.938200 18491 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.941365 18490 raft_consensus.cc:359] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.942003 18490 raft_consensus.cc:385] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.942270 18490 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.942860 18490 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.945595 18492 raft_consensus.cc:359] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.946245 18492 raft_consensus.cc:385] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.946491 18492 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.946878 18490 ts_tablet_manager.cc:1434] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:16:17.947715 18490 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:17.947438 18492 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.949756 18492 ts_tablet_manager.cc:1434] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 12:16:17.951222 18492 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:17.952982 18491 ts_tablet_manager.cc:1434] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:16:17.953691 18491 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:17.962791 18197 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.970618 18490 tablet_bootstrap.cc:654] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.977950 18491 tablet_bootstrap.cc:654] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.977950 18492 tablet_bootstrap.cc:654] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:17.982916 18490 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:17.983842 18490 ts_tablet_manager.cc:1403] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.006s
I20260709 12:16:17.987838 18490 raft_consensus.cc:359] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.988633 18490 raft_consensus.cc:385] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.988973 18490 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.989658 18490 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.991156 18492 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:17.991894 18492 ts_tablet_manager.cc:1403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.009s
I20260709 12:16:17.994670 18492 raft_consensus.cc:359] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.995188 18491 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:17.995507 18492 raft_consensus.cc:385] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.995702 18491 ts_tablet_manager.cc:1403] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.001s
I20260709 12:16:17.995860 18492 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.996603 18492 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.998277 18491 raft_consensus.cc:359] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:17.998896 18491 raft_consensus.cc:385] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:17.999162 18491 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:17.999755 18491 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.001003 18492 ts_tablet_manager.cc:1434] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:16:18.001056 18490 ts_tablet_manager.cc:1434] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:17.998438 18197 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.001889 18492 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.002025 18490 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.006070 18491 ts_tablet_manager.cc:1434] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:18.007853 18491 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.008483 18492 tablet_bootstrap.cc:654] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.014341 18491 tablet_bootstrap.cc:654] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.018033 18490 tablet_bootstrap.cc:654] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.026649 18491 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.027066 18491 ts_tablet_manager.cc:1403] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 12:16:18.030520 18491 raft_consensus.cc:359] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.031136 18491 raft_consensus.cc:385] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.031440 18491 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.032017 18491 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.032955 18490 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.033383 18490 ts_tablet_manager.cc:1403] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.032s	user 0.002s	sys 0.011s
I20260709 12:16:18.034122 18491 ts_tablet_manager.cc:1434] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:18.034965 18491 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.035173 18490 raft_consensus.cc:359] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.035888 18490 raft_consensus.cc:385] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.036217 18490 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.037051 18490 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.039708 18490 ts_tablet_manager.cc:1434] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:16:18.040814 18491 tablet_bootstrap.cc:654] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.040848 18490 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.043807 18667 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.044266 18667 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.045295 18550 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.046315 18667 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.047881 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.048489 18375 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:18.049923 18490 tablet_bootstrap.cc:654] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.053352 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.053354 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.054247 18447 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:18.054620 18613 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.056283 18613 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.056622 18613 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.065675 18613 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.066740 18490 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.067224 18490 ts_tablet_manager.cc:1403] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.027s	user 0.024s	sys 0.000s
I20260709 12:16:18.069227 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.069787 18492 tablet_bootstrap.cc:492] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.070246 18492 ts_tablet_manager.cc:1403] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.028s
I20260709 12:16:18.069993 18375 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.070062 18490 raft_consensus.cc:359] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.071033 18490 raft_consensus.cc:385] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.071378 18490 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.072085 18490 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.074337 18490 ts_tablet_manager.cc:1434] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:18.075688 18490 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.081939 18375 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.087055 18613 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.089581 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.090189 18490 tablet_bootstrap.cc:654] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.088316 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.091287 18667 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.092142 18447 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.092501 18667 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:18.093321 18667 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.094348 18550 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.099478 18492 raft_consensus.cc:359] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.100200 18492 raft_consensus.cc:385] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.100631 18492 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.101850 18491 tablet_bootstrap.cc:492] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.102169 18491 ts_tablet_manager.cc:1403] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.067s	user 0.010s	sys 0.009s
I20260709 12:16:18.103823 18492 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.116291 18492 ts_tablet_manager.cc:1434] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.003s
I20260709 12:16:18.117491 18492 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.121670 18491 raft_consensus.cc:359] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.122190 18491 raft_consensus.cc:385] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.122454 18491 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.123252 18491 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.126113 18491 ts_tablet_manager.cc:1434] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:16:18.127192 18491 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.132297 18667 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.132794 18667 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.134954 18491 tablet_bootstrap.cc:654] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.135804 18490 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.137485 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.138018 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.138816 18446 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:18.137950 18196 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.140235 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.141075 18613 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.141541 18613 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.141943 18613 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.142860 18667 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.136246 18490 ts_tablet_manager.cc:1403] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.061s	user 0.007s	sys 0.011s
I20260709 12:16:18.138131 18375 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:18.141872 18447 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.146095 18490 raft_consensus.cc:359] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.146823 18490 raft_consensus.cc:385] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.147158 18490 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.147882 18490 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.149474 18492 tablet_bootstrap.cc:654] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.150159 18490 ts_tablet_manager.cc:1434] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:16:18.151232 18490 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.159083 18490 tablet_bootstrap.cc:654] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.160712 18550 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.161164 18550 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.163154 18550 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.164681 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.165413 18296 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:18.167493 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.168588 18447 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:18.170264 18339 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:18.171348 18550 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.174649 18602 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.149309 18613 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.182695 18613 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.183293 18661 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.183771 18661 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.184335 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.184934 18372 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.185699 18661 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.192422 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.193130 18492 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.193579 18492 ts_tablet_manager.cc:1403] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.076s	user 0.022s	sys 0.018s
I20260709 12:16:18.194891 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.195154 18372 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.195561 18294 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
W20260709 12:16:18.196487 18339 leader_election.cc:343] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.197338 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:18.196275 18492 raft_consensus.cc:359] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.198020 18492 raft_consensus.cc:385] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.198308 18492 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.198904 18492 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.200057 18490 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.200585 18490 ts_tablet_manager.cc:1403] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.050s	user 0.024s	sys 0.009s
I20260709 12:16:18.200595 18492 ts_tablet_manager.cc:1434] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:18.197297 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.201682 18492 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.201908 18448 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.203424 18490 raft_consensus.cc:359] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.204090 18490 raft_consensus.cc:385] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.204363 18490 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.204965 18490 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.205870 18663 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.206251 18663 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.206570 18663 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.208751 18492 tablet_bootstrap.cc:654] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.213487 18663 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.216025 18490 ts_tablet_manager.cc:1434] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:16:18.216953 18490 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.219930 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:16:18.221503 18339 leader_election.cc:343] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.222059 18663 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.223032 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.225875 18613 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.227805 18602 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.231690 18296 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.234115 18667 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.253471 18490 tablet_bootstrap.cc:654] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.221998 18550 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.255761 18294 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.242852 18448 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.261402 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.262392 18667 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.263147 18667 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:18.266525 18613 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.267197 18613 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.267688 18296 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:18.269922 18613 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.271057 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:18.272390 18491 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.273221 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.275791 18491 ts_tablet_manager.cc:1403] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.009s
I20260709 12:16:18.277889 18663 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.278266 18446 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:18.278431 18663 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:18.286088 18670 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.286617 18670 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.277905 18667 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.291409 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.293275 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:18.296217 18613 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.300027 18491 raft_consensus.cc:359] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.304631 18491 raft_consensus.cc:385] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.307094 18670 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.307354 18491 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.308054 18491 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.310192 18491 ts_tablet_manager.cc:1434] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.000s
I20260709 12:16:18.316151 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.311095 18491 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.286788 18550 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
W20260709 12:16:18.323550 18264 leader_election.cc:343] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.327544 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.328805 18608 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.329746 18608 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.320394 18663 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.343009 18613 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.343918 18602 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.344437 18602 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.348220 18661 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:18.351696 18197 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.352900 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.353497 18616 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.353497 18446 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.356333 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.347375 18608 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.359380 18264 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.361959 18602 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.358175 18492 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.364773 18492 ts_tablet_manager.cc:1403] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.163s	user 0.035s	sys 0.060s
W20260709 12:16:18.367427 18266 leader_election.cc:343] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.368048 18266 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:18.369371 18670 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.370174 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.370826 18373 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:18.368144 18492 raft_consensus.cc:359] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.373804 18492 raft_consensus.cc:385] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.374239 18492 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.375444 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:18.429227 18492 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.433243 18492 ts_tablet_manager.cc:1434] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.002s
I20260709 12:16:18.433948 18492 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.435379 18490 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.435813 18490 ts_tablet_manager.cc:1403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.219s	user 0.020s	sys 0.009s
I20260709 12:16:18.431111 18544 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.437917 18608 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:18.434475 18659 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.442373 18608 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:18.442799 18608 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:18.443347 18491 tablet_bootstrap.cc:654] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.438277 18544 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.444113 18490 raft_consensus.cc:359] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.445071 18490 raft_consensus.cc:385] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.445349 18490 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.445941 18490 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:18.432248 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52193.3us
I20260709 12:16:18.456625 18658 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.457151 18658 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.460216 18658 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.444370 18659 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.463649 18659 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.465381 18490 ts_tablet_manager.cc:1434] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:16:18.466054 18490 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.447083 18544 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.484571 18663 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.485105 18663 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.492697 18492 tablet_bootstrap.cc:654] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.495025 18663 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.504515 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.505107 18375 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:18.506841 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.507432 18294 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:18.508562 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.509354 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.509698 18552 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.511612 18295 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:18.511715 18552 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.512111 18552 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.514669 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.514456 18608 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.515480 18300 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:18.510811 18197 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.517578 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.523636 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.523685 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.520229 18663 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:18.526583 18550 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:18.526904 18550 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:18.527691 18550 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.528568 18550 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:18.531397 18608 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.532060 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.536013 18415 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:18.536780 18748 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.536706 18661 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:18.537256 18748 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.537611 18748 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:18.539664 18415 leader_election.cc:343] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.539741 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.539860 18552 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.542315 18552 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.581393 18544 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.584980 18491 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.587993 18491 ts_tablet_manager.cc:1403] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.277s	user 0.020s	sys 0.014s
I20260709 12:16:18.587131 18490 tablet_bootstrap.cc:654] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.590078 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.593875 18300 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.594871 18491 raft_consensus.cc:359] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.596844 18491 raft_consensus.cc:385] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.597342 18491 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.603943 18300 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:18.606506 18492 tablet_bootstrap.cc:492] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.606947 18492 ts_tablet_manager.cc:1403] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.173s	user 0.008s	sys 0.044s
I20260709 12:16:18.607967 18748 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.608438 18375 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:18.609977 18748 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.610744 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:18.611876 18670 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:18.610613 18491 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.612071 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.612346 18670 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:18.612797 18373 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.613274 18670 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.614033 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.614921 18491 ts_tablet_manager.cc:1434] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.005s
I20260709 12:16:18.615021 18663 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:18.613312 18339 leader_election.cc:343] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.618436 18663 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:18.619441 18373 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:18.619377 18663 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.620481 18415 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:18.621426 18491 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.621481 18748 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:18.625286 18412 leader_election.cc:343] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:18.626392 18412 leader_election.cc:343] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.628574 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.631309 18412 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:18.631596 18375 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.635509 18749 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.637584 18375 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:18.638381 18616 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.636430 18670 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.638832 18616 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.639484 18670 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.640132 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:18.640882 18616 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.641466 18670 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.641597 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.626469 18544 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.642231 18294 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:18.642429 18659 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.642904 18659 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:18.643443 18492 raft_consensus.cc:359] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.643981 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.643795 18659 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.645011 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.644124 18492 raft_consensus.cc:385] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.649348 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.636909 18197 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.652518 18491 tablet_bootstrap.cc:654] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.654608 18195 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.656539 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.655437 18492 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.664537 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.665154 18300 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:18.667522 18339 leader_election.cc:343] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.646996 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.664757 18492 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.672724 18446 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:18.627053 18748 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:18.674469 18748 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.675499 18492 ts_tablet_manager.cc:1434] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.002s
I20260709 12:16:18.675881 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.676398 18492 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.676599 18608 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.647017 18616 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.659489 18670 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.677534 18616 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.677922 18616 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:18.680166 18412 leader_election.cc:343] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.686271 18616 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.687635 18375 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:18.688230 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.688822 18446 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:18.661479 18491 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.663570 18490 tablet_bootstrap.cc:492] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.677120 18608 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.689565 18608 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.689618 18490 ts_tablet_manager.cc:1403] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.224s	user 0.013s	sys 0.002s
I20260709 12:16:18.693042 18491 ts_tablet_manager.cc:1403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.072s	user 0.006s	sys 0.006s
I20260709 12:16:18.692494 18490 raft_consensus.cc:359] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.695935 18490 raft_consensus.cc:385] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.696270 18490 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.700608 18490 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.700352 18670 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:18.703116 18490 ts_tablet_manager.cc:1434] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:16:18.704602 18490 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.705694 18492 tablet_bootstrap.cc:654] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.709810 18300 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:18.710055 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.710716 18608 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.709802 18491 raft_consensus.cc:359] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.714423 18491 raft_consensus.cc:385] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.714645 18491 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.716147 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.716945 18375 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:18.713714 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.717855 18445 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.713683 18296 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:18.724901 18491 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.716861 18608 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.722316 18195 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.728317 18616 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.728855 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:18.730643 18445 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:18.737112 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:18.741084 18491 ts_tablet_manager.cc:1434] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 12:16:18.742033 18491 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.743489 18608 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.744313 18608 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:18.745152 18608 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.747677 18490 tablet_bootstrap.cc:654] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.748104 18657 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.748585 18657 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.750671 18657 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.753355 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.754029 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:18.756377 18264 leader_election.cc:343] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:18.757794 18266 leader_election.cc:343] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:18.758229 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:18.759256 18657 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.762080 18492 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.762564 18492 ts_tablet_manager.cc:1403] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.003s
I20260709 12:16:18.764629 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:18.765237 18492 raft_consensus.cc:359] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.765897 18492 raft_consensus.cc:385] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.766158 18492 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.766830 18490 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.766855 18552 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:18.771544 18490 ts_tablet_manager.cc:1403] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.000s
I20260709 12:16:18.766968 18492 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.774171 18492 ts_tablet_manager.cc:1434] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:16:18.774230 18490 raft_consensus.cc:359] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.775025 18490 raft_consensus.cc:385] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.775458 18491 tablet_bootstrap.cc:654] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.775410 18490 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.775091 18492 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.777045 18490 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.780398 18490 ts_tablet_manager.cc:1434] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
W20260709 12:16:18.780660 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:18.781229 18490 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.788120 18195 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:18.793846 18490 tablet_bootstrap.cc:654] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.795473 18492 tablet_bootstrap.cc:654] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.799784 18196 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.803992 18490 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.804479 18490 ts_tablet_manager.cc:1403] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.001s
I20260709 12:16:18.805976 18492 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.806535 18492 ts_tablet_manager.cc:1403] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.003s
I20260709 12:16:18.807004 18490 raft_consensus.cc:359] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.807659 18490 raft_consensus.cc:385] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.807930 18490 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.808548 18490 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.809778 18492 raft_consensus.cc:359] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.810415 18492 raft_consensus.cc:385] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.810565 18490 ts_tablet_manager.cc:1434] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:18.810673 18492 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.811468 18490 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.811417 18492 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.816190 18492 ts_tablet_manager.cc:1434] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:16:18.819751 18552 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:18.823467 18490 tablet_bootstrap.cc:654] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.829640 18490 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.830053 18490 ts_tablet_manager.cc:1403] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 12:16:18.831856 18608 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.832172 18492 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.832602 18490 raft_consensus.cc:359] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.833364 18490 raft_consensus.cc:385] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.833630 18490 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.834302 18490 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.836947 18490 ts_tablet_manager.cc:1434] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:18.837833 18490 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.841343 18491 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.841781 18491 ts_tablet_manager.cc:1403] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.100s	user 0.012s	sys 0.013s
I20260709 12:16:18.851076 18492 tablet_bootstrap.cc:654] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.851944 18490 tablet_bootstrap.cc:654] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:18.857316 18339 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.857360 18491 raft_consensus.cc:359] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.858103 18491 raft_consensus.cc:385] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.858489 18491 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.859311 18491 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.864367 18608 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:18.866253 18491 ts_tablet_manager.cc:1434] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:16:18.868300 18491 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.874254 18490 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.874655 18490 ts_tablet_manager.cc:1403] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.008s
I20260709 12:16:18.877058 18490 raft_consensus.cc:359] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.877677 18490 raft_consensus.cc:385] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.877954 18490 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.878562 18490 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.880033 18490 ts_tablet_manager.cc:1434] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:16:18.880864 18490 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.884397 18492 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.884888 18492 ts_tablet_manager.cc:1403] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.068s	user 0.008s	sys 0.005s
I20260709 12:16:18.889690 18492 raft_consensus.cc:359] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.890326 18492 raft_consensus.cc:385] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.890583 18492 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.891219 18492 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.897853 18491 tablet_bootstrap.cc:654] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.903213 18490 tablet_bootstrap.cc:654] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.905128 18492 ts_tablet_manager.cc:1434] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:16:18.906162 18492 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.910574 18748 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.911087 18748 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.913165 18748 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.914512 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.915167 18373 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:18.916922 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.917693 18296 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:18.919996 18492 tablet_bootstrap.cc:654] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.924021 18491 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.924404 18490 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:18.924443 18491 ts_tablet_manager.cc:1403] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.056s	user 0.003s	sys 0.012s
I20260709 12:16:18.924932 18490 ts_tablet_manager.cc:1403] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 12:16:18.926973 18491 raft_consensus.cc:359] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.927147 18490 raft_consensus.cc:359] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.927685 18491 raft_consensus.cc:385] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.927763 18490 raft_consensus.cc:385] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.927984 18491 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.928041 18490 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.929139 18491 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.929921 18490 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.927968 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.931542 18491 ts_tablet_manager.cc:1434] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:16:18.932554 18491 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.933902 18748 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.934267 18748 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.934584 18748 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.935767 18490 ts_tablet_manager.cc:1434] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:18.936584 18490 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:18.942159 18748 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.944940 18492 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.945380 18492 ts_tablet_manager.cc:1403] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:16:18.945437 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:18.946056 18296 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.947835 18492 raft_consensus.cc:359] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.948504 18492 raft_consensus.cc:385] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.948781 18492 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.949853 18748 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.950687 18491 tablet_bootstrap.cc:654] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.953637 18490 tablet_bootstrap.cc:654] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.949389 18492 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.958261 18296 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:18.959126 18492 ts_tablet_manager.cc:1434] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:18.960213 18492 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.961966 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.962800 18491 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:18.963304 18491 ts_tablet_manager.cc:1403] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 12:16:18.954612 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:18.964330 18373 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.965818 18491 raft_consensus.cc:359] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.966500 18491 raft_consensus.cc:385] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.966804 18491 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.967446 18491 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.968921 18491 ts_tablet_manager.cc:1434] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:16:18.966233 18492 tablet_bootstrap.cc:654] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.969750 18373 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:18.970569 18544 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:18.971063 18544 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:18.971890 18544 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.973493 18550 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.974063 18492 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:18.973937 18550 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.974495 18492 ts_tablet_manager.cc:1403] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 12:16:18.976109 18659 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:18.976646 18659 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.977438 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.977960 18296 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:18.977833 18492 raft_consensus.cc:359] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.978506 18492 raft_consensus.cc:385] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:18.978665 18550 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:18.978811 18492 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:18.979015 18659 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:18.979524 18492 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:18.980348 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:18.981431 18550 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:18.981537 18492 ts_tablet_manager.cc:1434] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:18.981802 18550 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:18.981966 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:18.982194 18550 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:18.982564 18449 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:18.983206 18492 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:18.984419 18491 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:18.985486 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:18.991748 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:18.992496 18491 tablet_bootstrap.cc:654] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:18.998272 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:19.002168 18197 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.001174 18659 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:19.004925 18490 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:19.005627 18490 ts_tablet_manager.cc:1403] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.002s
I20260709 12:16:19.007525 18492 tablet_bootstrap.cc:654] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.009001 18490 raft_consensus.cc:359] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.009689 18490 raft_consensus.cc:385] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.010008 18490 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.010937 18490 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.013447 18492 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:19.013891 18492 ts_tablet_manager.cc:1403] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:16:19.014114 18550 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.014741 18491 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:19.015142 18491 ts_tablet_manager.cc:1403] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.001s
I20260709 12:16:19.016356 18492 raft_consensus.cc:359] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.016748 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:19.017295 18294 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.017007 18492 raft_consensus.cc:385] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.017699 18492 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.018352 18492 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.020409 18492 ts_tablet_manager.cc:1434] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:19.021229 18492 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:19.021550 18491 raft_consensus.cc:359] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.022486 18491 raft_consensus.cc:385] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.022743 18491 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.023394 18490 ts_tablet_manager.cc:1434] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:16:19.023769 18491 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.025790 18491 ts_tablet_manager.cc:1434] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 12:16:19.026566 18491 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:19.027796 18492 tablet_bootstrap.cc:654] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.029588 18661 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.030028 18661 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.031806 18491 tablet_bootstrap.cc:654] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.032068 18661 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.023792 18294 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.038146 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.038949 18449 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.040730 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.041083 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.041946 18294 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.042693 18339 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:19.043534 18550 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.044291 18550 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:19.045332 18552 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.045809 18552 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.046218 18552 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.046463 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:19.047159 18449 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.056371 18552 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.059069 18552 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.059906 18550 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:19.061616 18659 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:19.065508 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:19.066253 18300 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.071888 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:19.072469 18445 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.065623 18550 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.085143 18490 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:19.094015 18296 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.111588 18448 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:19.130623 18544 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.135017 18552 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.131910 18544 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.138425 18552 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.140100 18544 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:19.145901 18300 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.133364 18659 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.172996 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.174105 18294 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:19.149204 18657 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.161621 18667 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.171576 18659 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.212926 18748 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.225816 18663 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.200212 18657 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.218040 18667 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.255394 18663 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.261592 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:19.266996 18412 leader_election.cc:343] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:19.242947 18748 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.271894 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.272662 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.294451 18783 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:19.295101 18784 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.291438 18552 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.295626 18784 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.286725 18295 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.298008 18445 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.298770 18375 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:19.305992 18659 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:19.309638 18552 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.310730 18491 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:19.310806 18492 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:19.311266 18491 ts_tablet_manager.cc:1403] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.285s	user 0.051s	sys 0.051s
I20260709 12:16:19.311631 18492 ts_tablet_manager.cc:1403] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.291s	user 0.066s	sys 0.041s
I20260709 12:16:19.314065 18784 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.286238 18449 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:19.320104 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47511.3us
I20260709 12:16:19.310267 18552 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.332347 18657 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.333729 18748 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:19.334403 18748 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.334977 18448 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:19.342303 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:19.329087 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.347186 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:19.348480 18667 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.330526 18663 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.358335 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:19.353410 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.344084 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.349649 18444 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.334990 18748 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.350106 18783 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:19.366707 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.371979 18298 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.373812 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.374485 18300 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.374341 18492 raft_consensus.cc:359] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.383509 18492 raft_consensus.cc:385] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.383796 18492 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.375543 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.377233 18788 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:19.382057 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:19.384387 18492 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.394338 18492 ts_tablet_manager.cc:1434] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.002s
I20260709 12:16:19.389294 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.394971 18659 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.404129 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.405249 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.395198 18447 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.408869 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.407732 18784 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.409659 18663 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.409785 18784 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.410126 18663 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.410205 18784 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.410594 18663 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.423771 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.424474 18446 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:19.400523 18667 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:16:19.429854 18490 tablet_bootstrap.cc:654] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.429574 18670 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:19.406689 18550 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.432732 18491 raft_consensus.cc:359] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.434232 18491 raft_consensus.cc:385] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.434486 18491 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.435669 18491 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:19.444247 18339 leader_election.cc:343] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:19.451560 18491 ts_tablet_manager.cc:1434] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.140s	user 0.005s	sys 0.001s
I20260709 12:16:19.435470 18550 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.452590 18491 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:19.453099 18264 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.458130 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:19.440696 18544 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:16:19.439633 18793 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 12:16:19.466442 18748 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:16:19.447368 18415 leader_election.cc:343] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:19.439824 18613 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:16:19.414836 18783 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.452735 18550 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.461201 18552 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.486481 18657 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.477999 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.501027 18492 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:19.490360 18608 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.466854 18442 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.476704 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:19.501860 18657 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.499862 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:19.520056 18339 leader_election.cc:343] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:19.526101 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 47013.5us
I20260709 12:16:19.521606 18657 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.522821 18544 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:19.511987 18608 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:19.527429 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 82207.2us
I20260709 12:16:19.542655 18795 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.549666 18795 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.567430 18659 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:19.569662 18663 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.575296 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.625438 18492 tablet_bootstrap.cc:654] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:19.577147 18415 leader_election.cc:343] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:19.630580 18491 tablet_bootstrap.cc:654] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.609647 18784 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.629571 18659 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:16:19.661446 18663 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.631302 18297 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:19.630198 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:19.663954 18296 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:19.629917 18550 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:19.633278 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 73971.4us
I20260709 12:16:19.633903 18748 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 12:16:19.693571 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:19.694403 18292 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.689512 18374 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:16:19.702966 18800 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:16:19.711833 18550 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.737735 18786 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 12:16:19.741494 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 80974.3us
I20260709 12:16:19.753788 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:19.753911 18784 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.754369 18295 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.754832 18490 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:19.755303 18490 ts_tablet_manager.cc:1403] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.679s	user 0.010s	sys 0.025s
I20260709 12:16:19.712466 18657 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.758646 18292 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.758781 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:19.763010 18297 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.766283 18657 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.767289 18300 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:16:19.771932 18266 leader_election.cc:343] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:19.773594 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:19.774328 18446 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.783375 18490 raft_consensus.cc:359] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.784334 18490 raft_consensus.cc:385] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:19.784557 18490 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:19.785203 18490 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.789682 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:19.790741 18608 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.797470 18195 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.805802 18490 ts_tablet_manager.cc:1434] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.002s
I20260709 12:16:19.803030 18295 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.816179 18297 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:19.840741 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:19.842882 18490 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:19.844847 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.849022 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.849743 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.841315 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:19.865090 18446 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:19.856594 18797 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.853564 18783 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:19.853278 18788 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:19.878605 18783 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:19.879909 18491 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:19.896484 18491 ts_tablet_manager.cc:1403] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.444s	user 0.043s	sys 0.048s
I20260709 12:16:19.874552 18797 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.909561 18798 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:19.910109 18798 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:19.895835 18788 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:19.917886 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:19.906602 18797 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.930037 18748 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:19.910951 18798 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.941406 18412 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:19.936786 18748 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:19.950438 18747 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:16:19.952129 18747 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:19.961947 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:19.962615 18795 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:19.972059 18264 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:19.974470 18795 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:19.976742 18813 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:16:19.974963 18795 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:19.981516 18787 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:19.982208 18787 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.985118 18787 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:19.978461 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:19.949175 18748 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:19.988433 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.008147 18491 raft_consensus.cc:359] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.008863 18491 raft_consensus.cc:385] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.009147 18491 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.011551 18491 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.945840 18783 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:19.986016 18788 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:20.015863 18811 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:16:19.999384 18441 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.039314 18490 tablet_bootstrap.cc:654] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:19.989738 18795 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.051837 18491 ts_tablet_manager.cc:1434] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.155s	user 0.007s	sys 0.014s
I20260709 12:16:20.054210 18790 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.054886 18790 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.032063 18616 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:16:20.097345 18490 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:20.097958 18490 ts_tablet_manager.cc:1403] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.255s	user 0.012s	sys 0.008s
I20260709 12:16:20.096266 18613 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.103875 18492 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:20.103622 18613 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.104321 18492 ts_tablet_manager.cc:1403] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.603s	user 0.028s	sys 0.042s
I20260709 12:16:20.105861 18613 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.088023 18790 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.125522 18803 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:20.124087 18661 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:16:20.139564 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.136962 18492 raft_consensus.cc:359] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.140576 18492 raft_consensus.cc:385] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.140967 18492 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.141816 18492 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.146939 18443 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:20.147783 18823 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 12:16:20.148234 18748 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.149231 18445 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:20.153228 18492 ts_tablet_manager.cc:1434] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.002s
I20260709 12:16:20.154047 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.155359 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:20.155969 18296 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 1: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:20.156065 18291 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:20.151355 18748 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:16:20.161967 18822 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:16:20.166632 18797 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.163503 18266 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:20.160095 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.168550 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.178857 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.182125 18368 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:20.179811 18613 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:20.218245 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 97773.9us
I20260709 12:16:20.180094 18441 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:20.226691 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.227396 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.193840 18442 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
W20260709 12:16:20.231395 18339 leader_election.cc:343] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.233748 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:20.234357 18297 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:20.234467 18266 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:20.234529 18661 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:20.229945 18368 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:20.236541 18613 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:20.236919 18797 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.231670 18264 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:20.238689 18490 raft_consensus.cc:359] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.239480 18490 raft_consensus.cc:385] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.239799 18490 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.241328 18490 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.243067 18667 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:20.244632 18667 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.245055 18667 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:20.248174 18491 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:20.237812 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.262979 18339 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:20.267588 18667 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:20.330862 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:20.314505 18491 tablet_bootstrap.cc:654] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.299206 18492 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
W20260709 12:16:20.331125 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 95581.4us
I20260709 12:16:20.316825 18415 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:20.343838 18748 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:20.344203 18748 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.344538 18748 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:20.326265 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.345268 18445 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:20.348109 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:20.348841 18613 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:20.349210 18613 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.349537 18613 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.351405 18748 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.352226 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:20.316126 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.353416 18296 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.353466 18819 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:20.353999 18748 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:20.292750 18490 ts_tablet_manager.cc:1434] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.194s	user 0.008s	sys 0.002s
I20260709 12:16:20.355476 18490 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
W20260709 12:16:20.312638 18834 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 12:16:20.315641 18836 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:16:20.357395 18613 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.360606 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.366518 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.366808 18492 tablet_bootstrap.cc:654] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.367324 18445 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.371409 18294 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:20.373764 18445 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:20.374874 18667 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.383381 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.384186 18445 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:20.375320 18613 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.279927 18797 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:20.385462 18797 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.385855 18797 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:20.389032 18296 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:20.389810 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.390419 18371 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
W20260709 12:16:20.377249 18266 leader_election.cc:343] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.379108 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:16:20.392403 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 120143us
I20260709 12:16:20.392920 18613 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:20.393405 18613 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:20.394163 18613 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.395830 18195 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.397616 18195 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.380892 18490 tablet_bootstrap.cc:654] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.381356 18837 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:20.407011 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.399199 18195 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.410837 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.411403 18367 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.412933 18491 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:20.413371 18491 ts_tablet_manager.cc:1403] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.165s	user 0.007s	sys 0.011s
I20260709 12:16:20.417057 18491 raft_consensus.cc:359] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.417689 18491 raft_consensus.cc:385] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.417956 18491 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.418134 18797 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.410666 18195 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.419322 18371 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:20.420538 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:20.422683 18492 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:20.423610 18799 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:20.424082 18799 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:20.424476 18797 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.423314 18492 ts_tablet_manager.cc:1403] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.124s	user 0.028s	sys 0.001s
I20260709 12:16:20.418723 18491 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.428674 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.435727 18491 ts_tablet_manager.cc:1434] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:16:20.437472 18491 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:20.424857 18799 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.434734 18790 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.441190 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.444132 18492 raft_consensus.cc:359] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.444684 18492 raft_consensus.cc:385] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.444911 18492 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.445437 18492 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.446190 18371 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:16:20.448545 18367 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:20.443457 18790 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:20.449581 18339 leader_election.cc:343] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.453035 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.452304 18415 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:20.469087 18492 ts_tablet_manager.cc:1434] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 12:16:20.569942 18296 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:20.465889 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:20.570724 18492 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:20.570945 18748 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:20.454298 18490 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:20.574782 18490 ts_tablet_manager.cc:1403] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.220s	user 0.007s	sys 0.013s
I20260709 12:16:20.453243 18790 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.577551 18490 raft_consensus.cc:359] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:20.572048 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119807us
I20260709 12:16:20.585440 18797 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:20.586920 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:16:20.587153 18338 leader_election.cc:343] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.592461 18198 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.594820 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.609338 18441 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:20.610991 18490 raft_consensus.cc:385] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.611367 18490 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.596438 18492 tablet_bootstrap.cc:654] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.597466 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.600764 18491 tablet_bootstrap.cc:654] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.610448 18795 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:20.619765 18797 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:20.620761 18490 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.634750 18339 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:20.619150 18448 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:20.639468 18490 ts_tablet_manager.cc:1434] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 12:16:20.640904 18795 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.641834 18490 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:20.641264 18795 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.643870 18795 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.645292 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:20.646915 18266 leader_election.cc:343] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.647360 18661 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:20.648048 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.648654 18799 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:20.649926 18264 leader_election.cc:343] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.650372 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:20.651686 18795 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:20.658892 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:20.663410 18837 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:20.663751 18837 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.664067 18837 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.664336 18491 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:20.664775 18491 ts_tablet_manager.cc:1403] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.227s	user 0.014s	sys 0.012s
I20260709 12:16:20.667280 18491 raft_consensus.cc:359] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.667487 18788 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:20.667930 18491 raft_consensus.cc:385] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.668443 18491 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.669660 18491 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.678653 18490 tablet_bootstrap.cc:654] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.675520 18803 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:20.684824 18491 ts_tablet_manager.cc:1434] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:16:20.685999 18491 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:20.689380 18195 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.699746 18492 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:20.700253 18492 ts_tablet_manager.cc:1403] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.005s
W20260709 12:16:20.702899 18339 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.711472 18491 tablet_bootstrap.cc:654] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.738412 18613 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:20.743886 18492 raft_consensus.cc:359] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.747538 18492 raft_consensus.cc:385] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.747925 18492 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
W20260709 12:16:20.744905 18266 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:20.745164 18837 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.754981 18492 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.759927 18492 ts_tablet_manager.cc:1434] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.002s
I20260709 12:16:20.761229 18492 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:20.763557 18793 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.764031 18793 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.766068 18793 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:20.769542 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:20.771780 18412 leader_election.cc:343] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.789772 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.791162 18371 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:20.792892 18195 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:20.794205 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:20.796579 18787 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:20.799633 18787 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.799980 18787 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.810130 18787 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.820664 18613 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:20.878345 18490 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:20.857301 18613 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:20.880272 18613 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.882273 18613 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:16:20.873944 18857 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:16:20.882938 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:20.879199 18490 ts_tablet_manager.cc:1403] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.239s	user 0.020s	sys 0.012s
W20260709 12:16:20.887786 18266 leader_election.cc:343] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.891664 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:20.892258 18441 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.893157 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:20.897243 18442 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:20.898672 18264 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:20.899647 18608 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:20.900005 18608 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:20.900315 18608 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.903090 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.906201 18491 tablet_bootstrap.cc:492] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:20.906654 18491 ts_tablet_manager.cc:1403] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.221s	user 0.020s	sys 0.010s
I20260709 12:16:20.890014 18787 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:16:20.907681 18412 leader_election.cc:343] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:20.911304 18491 raft_consensus.cc:359] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.912047 18491 raft_consensus.cc:385] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.912364 18491 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.832289 18837 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.914114 18491 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.916266 18491 ts_tablet_manager.cc:1434] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 12:16:20.917337 18491 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:20.919427 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.920025 18371 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.832976 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:20.931700 18441 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:20.944754 18299 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:20.942998 18491 tablet_bootstrap.cc:654] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.953423 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:20.955564 18492 tablet_bootstrap.cc:654] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:20.967456 18799 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:20.967947 18799 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:20.968713 18799 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.973402 18837 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:20.983243 18371 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:20.986001 18490 raft_consensus.cc:359] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.987926 18608 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:20.990612 18608 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:20.986980 18490 raft_consensus.cc:385] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:20.995708 18490 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:20.995142 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:20.996706 18490 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:20.997387 18266 leader_election.cc:343] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.001457 18801 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:21.002092 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:21.002666 18440 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.004004 18299 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:21.011950 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.019994 18806 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:21.020504 18806 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:21.021301 18491 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.021297 18806 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.022015 18491 ts_tablet_manager.cc:1403] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.001s
I20260709 12:16:21.035408 18491 raft_consensus.cc:359] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.036325 18491 raft_consensus.cc:385] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.036655 18491 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.037441 18491 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.038635 18490 ts_tablet_manager.cc:1434] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.154s	user 0.004s	sys 0.004s
W20260709 12:16:21.038741 18339 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:21.039737 18490 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.050437 18440 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:21.053720 18264 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:21.056313 18801 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:21.057593 18801 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:21.057178 18491 ts_tablet_manager.cc:1434] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.005s
I20260709 12:16:21.058403 18801 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.062474 18491 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.069537 18492 tablet_bootstrap.cc:492] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.069967 18492 ts_tablet_manager.cc:1403] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.309s	user 0.025s	sys 0.056s
I20260709 12:16:21.071841 18198 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:21.073493 18492 raft_consensus.cc:359] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.074290 18492 raft_consensus.cc:385] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.074556 18492 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.075366 18492 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.088428 18492 ts_tablet_manager.cc:1434] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:16:21.129892 18492 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.139161 18490 tablet_bootstrap.cc:654] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.144989 18195 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:21.156531 18795 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.165094 18795 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.169528 18795 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.171428 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:21.175549 18866 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:16:21.177359 18197 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:21.182750 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.183530 18371 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:21.185120 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:21.187629 18801 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.195497 18491 tablet_bootstrap.cc:654] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.196398 18801 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:21.198987 18492 tablet_bootstrap.cc:654] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.200057 18490 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.200537 18490 ts_tablet_manager.cc:1403] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.161s	user 0.022s	sys 0.002s
I20260709 12:16:21.204160 18490 raft_consensus.cc:359] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.205150 18490 raft_consensus.cc:385] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.205466 18490 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.206403 18490 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.210590 18492 tablet_bootstrap.cc:492] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.211050 18492 ts_tablet_manager.cc:1403] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.082s	user 0.008s	sys 0.004s
I20260709 12:16:21.213724 18797 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.214191 18797 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.216163 18797 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.217092 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:21.218492 18490 ts_tablet_manager.cc:1434] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:16:21.218770 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:21.220279 18338 leader_election.cc:343] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.214211 18492 raft_consensus.cc:359] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.221796 18492 raft_consensus.cc:385] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.222097 18492 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.224025 18371 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:21.224994 18339 leader_election.cc:343] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.222831 18492 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.227069 18490 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.225399 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.228171 18799 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:21.229185 18492 ts_tablet_manager.cc:1434] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:16:21.231689 18492 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.230811 18491 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.232314 18491 ts_tablet_manager.cc:1403] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.170s	user 0.015s	sys 0.007s
I20260709 12:16:21.241564 18490 tablet_bootstrap.cc:654] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.241708 18492 tablet_bootstrap.cc:654] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.245278 18491 raft_consensus.cc:359] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.245961 18491 raft_consensus.cc:385] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.246220 18491 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.246814 18491 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.254577 18491 ts_tablet_manager.cc:1434] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:16:21.255555 18491 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.259922 18491 tablet_bootstrap.cc:654] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.269034 18491 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.269455 18491 ts_tablet_manager.cc:1403] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:16:21.271955 18491 raft_consensus.cc:359] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.272629 18491 raft_consensus.cc:385] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.272892 18491 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.273435 18491 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.274072 18492 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.274495 18492 ts_tablet_manager.cc:1403] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.003s
I20260709 12:16:21.275430 18491 ts_tablet_manager.cc:1434] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:21.276464 18491 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.276634 18490 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.277077 18490 ts_tablet_manager.cc:1403] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.006s
I20260709 12:16:21.278157 18492 raft_consensus.cc:359] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.279640 18490 raft_consensus.cc:359] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.280287 18492 raft_consensus.cc:385] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.280701 18492 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.282073 18491 tablet_bootstrap.cc:654] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.281678 18492 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.283862 18492 ts_tablet_manager.cc:1434] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:16:21.284972 18492 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.280310 18490 raft_consensus.cc:385] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.288897 18490 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.289837 18490 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.292094 18490 ts_tablet_manager.cc:1434] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:16:21.292932 18490 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.294998 18492 tablet_bootstrap.cc:654] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.299095 18491 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.299556 18491 ts_tablet_manager.cc:1403] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.001s
I20260709 12:16:21.302125 18491 raft_consensus.cc:359] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.302768 18491 raft_consensus.cc:385] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.303007 18491 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.303572 18793 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.303699 18491 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.304054 18793 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.307467 18491 ts_tablet_manager.cc:1434] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:21.307479 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.308379 18491 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
W20260709 12:16:21.309249 18415 leader_election.cc:343] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.309464 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.313990 18793 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.314253 18491 tablet_bootstrap.cc:654] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.317898 18492 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.318344 18492 ts_tablet_manager.cc:1403] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 12:16:21.321468 18492 raft_consensus.cc:359] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.322137 18492 raft_consensus.cc:385] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.322412 18492 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.323094 18492 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.325160 18492 ts_tablet_manager.cc:1434] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:21.326165 18492 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
W20260709 12:16:21.326602 18412 leader_election.cc:343] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.327061 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.333583 18806 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:21.349129 18490 tablet_bootstrap.cc:654] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.353705 18492 tablet_bootstrap.cc:654] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.357249 18490 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.357983 18490 ts_tablet_manager.cc:1403] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.003s
I20260709 12:16:21.358913 18491 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.359386 18491 ts_tablet_manager.cc:1403] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.019s
I20260709 12:16:21.360178 18492 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.360600 18492 ts_tablet_manager.cc:1403] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 12:16:21.362046 18491 raft_consensus.cc:359] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.362859 18491 raft_consensus.cc:385] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.363116 18491 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.363262 18492 raft_consensus.cc:359] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.363991 18492 raft_consensus.cc:385] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.363788 18491 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.364243 18492 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.364996 18492 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.366147 18491 ts_tablet_manager.cc:1434] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:21.368494 18491 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.370421 18492 ts_tablet_manager.cc:1434] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:16:21.372648 18492 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.374032 18491 tablet_bootstrap.cc:654] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.387838 18492 tablet_bootstrap.cc:654] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:21.395051 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:21.367743 18490 raft_consensus.cc:359] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.396704 18806 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.397012 18490 raft_consensus.cc:385] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.397444 18490 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.397238 18806 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.398336 18490 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.399567 18806 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.400831 18490 ts_tablet_manager.cc:1434] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:16:21.402034 18490 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.404798 18491 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.405295 18491 ts_tablet_manager.cc:1403] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.009s
I20260709 12:16:21.405107 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.404803 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.406912 18295 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
W20260709 12:16:21.407306 18415 leader_election.cc:343] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.408190 18490 tablet_bootstrap.cc:654] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.407857 18491 raft_consensus.cc:359] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.408280 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:21.409706 18806 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:21.410070 18806 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.410446 18806 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.408541 18491 raft_consensus.cc:385] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.412007 18491 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.415046 18492 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.415697 18492 ts_tablet_manager.cc:1403] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.006s
I20260709 12:16:21.419705 18492 raft_consensus.cc:359] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.420640 18492 raft_consensus.cc:385] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.420977 18492 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.421762 18492 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.422763 18491 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.426098 18491 ts_tablet_manager.cc:1434] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 12:16:21.427302 18491 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.429749 18787 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:21.436275 18492 ts_tablet_manager.cc:1434] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:16:21.437158 18492 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.442534 18492 tablet_bootstrap.cc:654] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.446527 18490 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.446950 18490 ts_tablet_manager.cc:1403] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.015s
W20260709 12:16:21.450323 18266 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:21.450718 18491 tablet_bootstrap.cc:654] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.452705 18793 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.453150 18793 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.454017 18490 raft_consensus.cc:359] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.454852 18490 raft_consensus.cc:385] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.455147 18490 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.456074 18490 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.455157 18793 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.457996 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.460768 18492 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.461185 18492 ts_tablet_manager.cc:1403] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 12:16:21.460229 18490 ts_tablet_manager.cc:1434] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:16:21.462219 18490 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.463575 18492 raft_consensus.cc:359] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.464196 18492 raft_consensus.cc:385] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.464457 18492 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.465059 18492 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.468664 18490 tablet_bootstrap.cc:654] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.468896 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.470536 18786 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.470978 18786 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.473651 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.474252 18374 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:21.475793 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:21.477487 18492 ts_tablet_manager.cc:1434] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:21.477571 18795 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:21.478094 18795 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.478452 18795 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.479835 18490 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.480424 18490 ts_tablet_manager.cc:1403] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:16:21.480067 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.478523 18492 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.481422 18793 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:21.482993 18550 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:21.483201 18490 raft_consensus.cc:359] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.483919 18490 raft_consensus.cc:385] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.484233 18490 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.485047 18793 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.484843 18490 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.485502 18793 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.486743 18492 tablet_bootstrap.cc:654] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.486835 18490 ts_tablet_manager.cc:1434] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:16:21.487813 18490 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.489001 18786 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.490483 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:21.491871 18264 leader_election.cc:343] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.492363 18793 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.494760 18795 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.495543 18490 tablet_bootstrap.cc:654] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.496170 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.496785 18374 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:21.496192 18793 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.497677 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:21.497860 18793 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.498474 18787 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:21.498829 18787 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.499176 18787 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.499996 18793 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.502177 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:21.502722 18374 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.506584 18490 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.507031 18490 ts_tablet_manager.cc:1403] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:16:21.507206 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.518074 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.518410 18492 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.518386 18490 raft_consensus.cc:359] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.519006 18492 ts_tablet_manager.cc:1403] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.004s
I20260709 12:16:21.519109 18490 raft_consensus.cc:385] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.519063 18295 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:21.519354 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.519634 18490 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.520295 18490 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.523265 18492 raft_consensus.cc:359] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.523993 18492 raft_consensus.cc:385] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.524296 18492 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.526537 18786 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.527072 18786 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.527125 18492 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.442430 18806 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.537456 18492 ts_tablet_manager.cc:1434] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:16:21.523054 18787 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.545084 18492 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.545533 18806 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.547122 18787 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.547823 18893 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:21.547937 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:21.548791 18490 ts_tablet_manager.cc:1434] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 12:16:21.549155 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:21.549850 18490 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:21.549775 18367 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.555025 18492 tablet_bootstrap.cc:654] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.560678 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:21.561342 18442 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:21.565110 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:21.556128 18795 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.572693 18492 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:21.573143 18492 ts_tablet_manager.cc:1403] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.005s
I20260709 12:16:21.573783 18801 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:21.574184 18801 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.574512 18801 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.575568 18492 raft_consensus.cc:359] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.576217 18492 raft_consensus.cc:385] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.510246 18374 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:21.579663 18491 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.580149 18491 ts_tablet_manager.cc:1403] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.153s	user 0.026s	sys 0.025s
I20260709 12:16:21.580188 18799 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:21.582511 18491 raft_consensus.cc:359] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.558697 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.583566 18371 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:21.557221 18367 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:21.559303 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:21.559473 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.576484 18492 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.584705 18448 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.585093 18492 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.585242 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:21.584698 18491 raft_consensus.cc:385] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.586179 18491 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
W20260709 12:16:21.559830 18415 leader_election.cc:343] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.586746 18491 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.587558 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:21.588200 18491 ts_tablet_manager.cc:1434] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:21.588155 18295 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.591064 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:21.591614 18299 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.592271 18442 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.588150 18608 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:21.593559 18608 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:21.594365 18608 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.598439 18295 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:21.598778 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:21.620697 18891 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:21.621243 18891 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:21.621924 18898 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:21.622148 18891 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.623867 18550 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.599294 18490 tablet_bootstrap.cc:654] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.624316 18550 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.624720 18448 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:21.602049 18445 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.627322 18550 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.582435 18801 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.609251 18491 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:21.606040 18299 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:21.633412 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.612787 18492 ts_tablet_manager.cc:1434] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
W20260709 12:16:21.635924 18338 leader_election.cc:343] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:21.636667 18492 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:21.644057 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:21.644788 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:21.645022 18892 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:21.645460 18448 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 0.
I20260709 12:16:21.645591 18892 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:21.647734 18892 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.653131 18492 tablet_bootstrap.cc:654] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.656214 18491 tablet_bootstrap.cc:654] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:21.557862 18786 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.656777 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:21.657364 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:21.657954 18371 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.654095 18801 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.658428 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:21.662089 18440 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.658435 18661 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:21.667894 18661 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:21.668565 18893 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.668360 18661 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:21.675376 18550 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.675980 18550 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.673879 18371 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:21.685958 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:21.695920 18550 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.702564 18608 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.703824 18608 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.706658 18608 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:21.669081 18893 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.786917 18608 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.828140 18608 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.721901 18197 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:21.744402 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:21.856472 18795 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:21.870846 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:21.878067 18448 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:21.881978 18440 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:21.868008 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:21.875790 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:21.883745 18893 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:21.909781 18491 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:21.911731 18491 ts_tablet_manager.cc:1403] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.303s	user 0.027s	sys 0.013s
I20260709 12:16:21.879791 18795 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.927167 18490 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:21.927862 18490 ts_tablet_manager.cc:1403] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.378s	user 0.018s	sys 0.012s
I20260709 12:16:21.894593 18292 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 0.
I20260709 12:16:21.938337 18491 raft_consensus.cc:359] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:21.939327 18491 raft_consensus.cc:385] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:21.939726 18491 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:21.919988 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:22.010561 18490 raft_consensus.cc:359] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.013198 18266 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:22.013267 18490 raft_consensus.cc:385] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:22.013859 18490 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:22.014180 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:22.014810 18490 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:21.989301 18900 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:16:22.012466 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:22.035580 18786 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:22.036767 18801 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:22.093562 18608 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.108340 18786 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:22.123361 18795 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.131852 18801 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:22.193706 18786 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.242726 18491 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:22.251394 18415 leader_election.cc:343] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:22.242192 18661 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.002s
I20260709 12:16:22.280490 18491 ts_tablet_manager.cc:1434] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.368s	user 0.020s	sys 0.011s
I20260709 12:16:22.287497 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:22.289880 18550 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:22.303256 18490 ts_tablet_manager.cc:1434] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.375s	user 0.014s	sys 0.047s
I20260709 12:16:22.277853 18661 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:22.307118 18901 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
W20260709 12:16:22.303267 18897 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:16:22.313298 18491 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:22.318946 18661 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.344965 18793 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:22.358739 18492 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:22.359206 18492 ts_tablet_manager.cc:1403] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.723s	user 0.026s	sys 0.051s
I20260709 12:16:22.297060 18801 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:22.360334 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53733.5us
W20260709 12:16:22.367542 18917 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:16:22.430529 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:22.433714 18297 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.433879 18922 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:22.434432 18922 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.441915 18922 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.446746 18297 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:16:22.481915 18786 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.482991 18491 tablet_bootstrap.cc:654] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:22.492690 18492 raft_consensus.cc:359] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.494493 18492 raft_consensus.cc:385] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:22.494866 18492 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:22.495886 18492 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.499173 18492 ts_tablet_manager.cc:1434] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.000s
I20260709 12:16:22.500142 18492 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:22.498991 18786 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.505192 18895 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:22.505947 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53500.9us
I20260709 12:16:22.509837 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:22.511807 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:22.513774 18891 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:22.514114 18891 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:22.514438 18891 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.515492 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:22.548950 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:22.552474 18550 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:22.552944 18550 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:22.553653 18550 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:22.558189 18266 leader_election.cc:343] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:22.559624 18266 leader_election.cc:343] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:22.562856 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:22.563592 18300 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:22.555830 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:22.562100 18795 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:22.779839 18490 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:22.760049 18447 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:22.808849 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:22.823526 18443 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.826188 18938 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:16:22.822933 18891 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.854206 18492 tablet_bootstrap.cc:654] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:22.861701 18367 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.863976 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:22.864751 18449 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:22.868793 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:22.869836 18444 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:22.772146 18795 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.876709 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:22.826874 18938 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:22.958635 18938 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.960107 18795 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:22.980561 18443 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:22.971716 18891 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:22.983974 18449 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
W20260709 12:16:23.031797 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 247296us
I20260709 12:16:23.051688 18373 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.055066 18490 tablet_bootstrap.cc:654] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:23.061484 18893 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:23.066239 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.068135 18371 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.079041 18923 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:16:23.080123 18923 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.081589 18915 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.092350 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:23.087097 18491 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:23.094523 18491 ts_tablet_manager.cc:1403] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.781s	user 0.107s	sys 0.141s
I20260709 12:16:23.102438 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.096276 18893 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.110623 18492 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:23.117379 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.118196 18442 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:23.120952 18898 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:23.121438 18898 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:23.121897 18898 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.114141 18491 raft_consensus.cc:359] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:23.128890 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50910.8us
I20260709 12:16:23.125794 18491 raft_consensus.cc:385] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.129587 18492 ts_tablet_manager.cc:1403] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.630s	user 0.012s	sys 0.005s
I20260709 12:16:23.129890 18491 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.139901 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:23.140637 18375 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:23.141855 18197 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 103) took 1429 ms. Trace:
I20260709 12:16:23.145247 18923 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.145913 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.148726 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:23.149614 18300 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.155650 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.155934 18195 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:16:23.157733 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:23.158900 18923 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:23.159967 18923 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:23.160938 18198 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 102) took 1149 ms. Trace:
I20260709 12:16:23.161326 18198 rpcz_store.cc:276] 0709 12:16:22.011849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:22.012035 (+   186us) service_pool.cc:224] Handling call
0709 12:16:23.160741 (+1148706us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260709 12:16:23.167711 18491 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.161283 18923 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.187338 18491 ts_tablet_manager.cc:1434] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.092s	user 0.019s	sys 0.007s
I20260709 12:16:23.189319 18491 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Bootstrap starting.
I20260709 12:16:23.191778 18910 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:23.142321 18197 rpcz_store.cc:276] 0709 12:16:21.712026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:21.712212 (+   186us) service_pool.cc:224] Handling call
0709 12:16:23.141579 (+1429367us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:22.088540 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:22.089342 (+   802us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:22.089388 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:16:22.089414 (+    26us) tablet.cc:662] Decoding operations
0709 12:16:22.089868 (+   454us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:22.089944 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:22.089985 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:22.090225 (+   240us) tablet.cc:701] Row locks acquired
0709 12:16:22.090245 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:16:22.090446 (+   201us) write_op.cc:270] Start()
0709 12:16:22.090587 (+   141us) write_op.cc:276] Timestamp: P: 1783599382090388 usec, L: 0
0709 12:16:22.090617 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:16:22.122388 (+ 31771us) log.cc:844] Serialized 540 byte log entry
0709 12:16:22.355977 (+233589us) op_driver.cc:464] REPLICATION: finished
0709 12:16:22.503723 (+147746us) write_op.cc:301] APPLY: starting
0709 12:16:22.503796 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:22.543463 (+ 39667us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:22.568096 (+ 24633us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:22.750074 (+181978us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:22.750243 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:16:22.757127 (+  6884us) write_op.cc:312] APPLY: finished
0709 12:16:22.763086 (+  5959us) log.cc:844] Serialized 22 byte log entry
0709 12:16:22.763323 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:22.763434 (+   111us) write_op.cc:454] Released schema lock
0709 12:16:22.763494 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:22.923194 (+159700us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b54003ebda8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6f2389e1 00007f3c6f23764c 00007f3c6c8c83fd 00007f3c6c8b4389 00007f3c6c8b3b53 00007f3c6c8b7d61 00007f3c6c8b7d19 00007f3c6c8b7ca9 00007f3c6c8b7c71 00007f3c6c8b6e9d 0000000000388634
0709 12:16:22.925506 (+  2312us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:22.925555 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2379,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":147525,"apply.run_cpu_time_us":15042,"apply.run_wall_time_us":421723,"num_ops":1,"prepare.queue_time_us":239695,"prepare.run_cpu_time_us":3566,"prepare.run_wall_time_us":105762,"replication_time_us":265210,"spinlock_wait_cycles":10273152,"thread_start_us":293985,"threads_started":2}]]}
I20260709 12:16:23.192528 18910 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.196199 18893 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:23.202049 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:16:23.205619 18913 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:23.209056 18264 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:23.210755 18926 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:16:23.211120 18926 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:23.211477 18926 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:23.207993 18371 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:23.221170 18795 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.224663 18910 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:23.235654 18492 raft_consensus.cc:359] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.237082 18492 raft_consensus.cc:385] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.237265 18492 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.241029 18492 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.243818 18492 ts_tablet_manager.cc:1434] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.110s	user 0.006s	sys 0.001s
I20260709 12:16:23.248049 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.248504 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.244488 18935 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.262467 18939 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:23.266285 18338 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:16:23.266774 18952 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:16:23.269626 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:23.270360 18368 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:23.273677 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:23.271543 18938 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:23.277776 18492 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Bootstrap starting.
I20260709 12:16:23.277084 18913 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:23.281790 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:23.282757 18490 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:23.283414 18490 ts_tablet_manager.cc:1403] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.504s	user 0.027s	sys 0.014s
I20260709 12:16:23.296586 18910 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.309268 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.311154 18910 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.315377 18913 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:23.315900 18913 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.316226 18898 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.317420 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:23.323042 18933 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:23.284471 18926 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.325466 18898 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:16:23.321952 18246 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:23.327900 18490 raft_consensus.cc:359] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.328688 18490 raft_consensus.cc:385] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.328984 18490 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.329115 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:23.329897 18490 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.329854 18898 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:23.330529 18440 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:23.336385 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49379.1us
I20260709 12:16:23.340010 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:23.340765 18300 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:23.347643 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:23.350041 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:23.350306 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.351368 18490 ts_tablet_manager.cc:1434] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.004s
I20260709 12:16:23.369949 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.370829 18368 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:16:23.375747 18266 leader_election.cc:343] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:23.376631 18266 leader_election.cc:343] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:23.357051 18913 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.383033 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.383805 18446 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:23.385620 18490 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:23.386951 18264 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.389581 18935 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:23.397706 18926 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.397984 18935 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:23.398404 18935 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.402740 18368 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:16:23.409559 18895 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.418116 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:23.418535 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:23.419549 18895 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:23.418198 18933 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:23.420128 18895 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:23.420562 18933 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.425290 18916 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:23.431982 18933 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.431043 18916 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:23.448666 18491 tablet_bootstrap.cc:654] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:23.468977 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:23.469808 18441 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:23.506590 18492 tablet_bootstrap.cc:654] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:23.506947 18954 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:16:23.474220 18916 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.513994 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.515336 18440 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:23.482055 18895 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.520010 18935 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:23.545918 18415 leader_election.cc:343] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:23.535964 18933 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:16:23.549477 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.554241 18490 tablet_bootstrap.cc:654] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:23.546949 18415 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:23.565344 18441 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:16:23.565621 18935 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.567123 18492 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: No bootstrap required, opened a new log
I20260709 12:16:23.567616 18492 ts_tablet_manager.cc:1403] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Time spent bootstrapping tablet: real 0.323s	user 0.007s	sys 0.010s
I20260709 12:16:23.561054 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:23.571332 18964 raft_consensus.cc:2764] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:16:23.572597 18895 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:23.600737 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.605522 18926 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:23.614308 18492 raft_consensus.cc:359] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.617069 18492 raft_consensus.cc:385] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.617386 18492 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.620658 18491 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: No bootstrap required, opened a new log
I20260709 12:16:23.621140 18491 ts_tablet_manager.cc:1403] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Time spent bootstrapping tablet: real 0.432s	user 0.037s	sys 0.026s
I20260709 12:16:23.612176 18926 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:23.628032 18492 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:23.634517 18960 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.151s	user 0.003s	sys 0.000s
I20260709 12:16:23.645857 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:23.648847 18926 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.649232 18444 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.649278 18492 ts_tablet_manager.cc:1434] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.003s
I20260709 12:16:23.677435 18444 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:23.688905 18491 raft_consensus.cc:359] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.689707 18491 raft_consensus.cc:385] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.689993 18491 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.701009 18491 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.706005 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:23.707330 18293 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:16:23.708022 18490 tablet_bootstrap.cc:492] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:23.720623 18490 ts_tablet_manager.cc:1403] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.335s	user 0.009s	sys 0.024s
W20260709 12:16:23.718900 18266 leader_election.cc:343] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:23.740638 18198 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:16:23.742823 18490 raft_consensus.cc:359] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.743566 18490 raft_consensus.cc:385] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.743846 18490 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.747486 18926 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:23.752063 18914 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:23.752748 18914 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.754994 18490 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.755422 18914 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.758697 18490 ts_tablet_manager.cc:1434] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:16:23.759690 18490 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:23.765193 18491 ts_tablet_manager.cc:1434] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Time spent starting tablet: real 0.144s	user 0.005s	sys 0.001s
I20260709 12:16:23.765614 18490 tablet_bootstrap.cc:654] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:23.773476 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:16:23.788478 18266 leader_election.cc:343] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:23.788259 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:23.762430 18198 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:23.787307 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 145481us
I20260709 12:16:23.797828 18975 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.820155 18490 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:23.820910 18490 ts_tablet_manager.cc:1403] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.010s
I20260709 12:16:23.825906 18264 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.828140 18916 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:23.829246 18916 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:23.830096 18916 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.833359 18801 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.845680 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.846388 18448 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 0.
I20260709 12:16:23.833993 18975 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:23.853382 18197 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.857460 18490 raft_consensus.cc:359] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.858172 18490 raft_consensus.cc:385] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.858465 18490 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.859120 18490 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.879160 18490 ts_tablet_manager.cc:1434] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:16:23.880265 18490 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Bootstrap starting.
I20260709 12:16:23.894385 18490 tablet_bootstrap.cc:654] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:23.905747 18264 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:23.906621 18914 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:23.907006 18914 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:23.907385 18914 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.911584 18490 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: No bootstrap required, opened a new log
I20260709 12:16:23.912211 18490 ts_tablet_manager.cc:1403] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 12:16:23.915546 18490 raft_consensus.cc:359] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.917083 18490 raft_consensus.cc:385] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:23.917452 18490 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Initialized, Role: FOLLOWER
I20260709 12:16:23.918469 18975 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:23.917950 18198 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:23.918332 18490 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.918957 18975 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.919593 18914 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.919767 18198 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:16:23.921337 18490 ts_tablet_manager.cc:1434] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:23.921604 18914 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.922484 18975 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:23.922755 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:23.923327 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:23.923734 18446 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:23.924031 18373 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.933596 18976 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:23.936623 18373 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:23.938254 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:23.939390 18974 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:23.939992 18974 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:23.941001 18974 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:23.944432 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:23.945147 18446 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.954238 18446 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:16:23.969434 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:23.975852 18891 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:23.979324 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:23.981177 18920 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:23.996649 18920 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:24.001675 18938 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:24.002192 18938 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:24.004307 18938 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:24.005838 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:24.006738 18440 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 1: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:16:24.011664 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:24.013177 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:24.014765 18939 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:24.026492 18367 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 12:16:24.026619 18939 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:24.033490 18926 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.037844 18197 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 3, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.039456 18197 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.041787 18197 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.073733 18440 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 12:16:24.077045 18801 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.097406 18196 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.123664 18926 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.132809 18974 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.359687 18801 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.374104 18926 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.405653 18974 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:24.417886 18914 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:25.153066 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:25.185328 19000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:25.186446 18998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:25.186463 18997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:25.187793 17157 server_base.cc:1061] running on GCE node
I20260709 12:16:25.189263 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:25.189534 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:25.189786 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599385189766 us; error 0 us; skew 500 ppm
I20260709 12:16:25.190568 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:25.194067 17157 webserver.cc:533] Webserver started at http://127.16.193.68:46303/ using document root <none> and password file <none>
I20260709 12:16:25.194736 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:25.194988 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:25.195346 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:25.196477 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root/instance:
uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
format_stamp: "Formatted at 2026-07-09 12:16:25 on dist-test-slave-n640"
I20260709 12:16:25.203809 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:25.208362 19005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:25.209489 17157 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:16:25.209858 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
format_stamp: "Formatted at 2026-07-09 12:16:25 on dist-test-slave-n640"
I20260709 12:16:25.210193 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:25.233480 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:25.234759 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:25.236270 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:25.238670 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:25.238940 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:25.239193 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:25.239444 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:25.293104 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.68:43821
I20260709 12:16:25.293207 19068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.68:43821 every 8 connection(s)
I20260709 12:16:25.297163 17157 auto_leader_rebalancer-test.cc:439] add a tserver: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:25.314708 19069 heartbeater.cc:344] Connected to a master server at 127.16.193.126:46027
I20260709 12:16:25.315142 19069 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:25.315955 19069 heartbeater.cc:507] Master 127.16.193.126:46027 requested a full tablet report, sending...
I20260709 12:16:25.317992 18197 ts_manager.cc:194] Registered new tserver with Master: 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:25.319511 18197 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41430
I20260709 12:16:25.465480 18440 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.474391 18373 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.474529 18297 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.476048 19076 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.479089 18890 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.488107 18891 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:16:25.492288 18412 consensus_peers.cc:597] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d266f68528964b1981ffe1df49861b08. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:25.491194 18373 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.511608 18197 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.532212 18297 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.545889 18295 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.560669 18440 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.562054 19088 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.568953 18373 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.570389 19088 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:25.579433 18263 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ab0a3f9b34149918a57aec5d20b4884. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:25.583726 18914 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.585542 18373 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.588393 18440 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.599498 18446 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.607647 18195 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.610265 18373 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:25.610874 18412 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 389aa517d744435bb043ff8c52d86c2d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:25.611490 18295 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.611588 18891 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.613533 18890 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.630206 19076 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.632089 18297 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.632436 18373 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.651268 18197 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.673178 18373 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.681887 18446 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.683380 18920 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.684211 18297 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:25.685434 18939 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:25.697754 18338 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80bc05a330c64471aaa556af75bea031. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:25.699268 18920 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.704397 18297 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.711352 18440 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.721583 18197 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:25.897348 19100 ts_tablet_manager.cc:933] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:25.899188 19100 tablet_copy_client.cc:323] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:25.912688 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet d266f68528964b1981ffe1df49861b08 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:25.913295 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet d266f68528964b1981ffe1df49861b08 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-d266f68528964b1981ffe1df49861b08
I20260709 12:16:25.921638 18459 tablet_copy_source_session.cc:215] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:25.925609 19100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d266f68528964b1981ffe1df49861b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:25.941758 19100 tablet_copy_client.cc:806] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:25.942391 19100 tablet_copy_client.cc:670] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:25.947058 19100 tablet_copy_client.cc:538] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:25.955917 19100 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:25.964037 19103 ts_tablet_manager.cc:933] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:25.967491 19103 tablet_copy_client.cc:323] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:25.979740 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 6ab0a3f9b34149918a57aec5d20b4884 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:25.980388 19100 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:25.980404 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 6ab0a3f9b34149918a57aec5d20b4884 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-6ab0a3f9b34149918a57aec5d20b4884
I20260709 12:16:25.981611 19100 tablet_bootstrap.cc:492] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:25.982368 19100 ts_tablet_manager.cc:1403] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.027s	user 0.023s	sys 0.000s
I20260709 12:16:25.985546 19100 raft_consensus.cc:359] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.986439 19100 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:25.987133 19100 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:25.988972 18310 tablet_copy_source_session.cc:215] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:25.992172 19069 heartbeater.cc:499] Master 127.16.193.126:46027 was elected leader, sending a full tablet report...
I20260709 12:16:25.995820 19107 ts_tablet_manager.cc:933] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:25.997304 19100 ts_tablet_manager.cc:1434] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:16:25.999155 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-d266f68528964b1981ffe1df49861b08 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:25.999677 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-d266f68528964b1981ffe1df49861b08 on tablet d266f68528964b1981ffe1df49861b08 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:26.000844 19103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ab0a3f9b34149918a57aec5d20b4884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:26.006980 19107 tablet_copy_client.cc:323] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:26.009074 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 389aa517d744435bb043ff8c52d86c2d from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:26.009673 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 389aa517d744435bb043ff8c52d86c2d from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-389aa517d744435bb043ff8c52d86c2d
I20260709 12:16:26.017604 19103 tablet_copy_client.cc:806] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:26.018049 18459 tablet_copy_source_session.cc:215] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:26.018462 19103 tablet_copy_client.cc:670] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:26.022322 19103 tablet_copy_client.cc:538] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:26.023047 19107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389aa517d744435bb043ff8c52d86c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:26.034189 19103 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:26.039323 19107 tablet_copy_client.cc:806] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:26.039927 19107 tablet_copy_client.cc:670] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:26.044219 19107 tablet_copy_client.cc:538] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:26.060596 19107 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:26.067749 19103 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:26.069490 19103 tablet_bootstrap.cc:492] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:26.070205 19103 ts_tablet_manager.cc:1403] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.011s
I20260709 12:16:26.073454 19103 raft_consensus.cc:359] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.074580 19103 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:26.075302 19103 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.077725 19103 ts_tablet_manager.cc:1434] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:26.079751 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-6ab0a3f9b34149918a57aec5d20b4884 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:26.080251 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-6ab0a3f9b34149918a57aec5d20b4884 on tablet 6ab0a3f9b34149918a57aec5d20b4884 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:26.089313 19107 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:26.090255 19107 tablet_bootstrap.cc:492] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:26.092077 19107 ts_tablet_manager.cc:1403] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.008s
I20260709 12:16:26.094964 19107 raft_consensus.cc:359] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.096356 19107 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:26.097040 19107 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.098979 19107 ts_tablet_manager.cc:1434] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:26.100857 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-389aa517d744435bb043ff8c52d86c2d received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:26.101261 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-389aa517d744435bb043ff8c52d86c2d on tablet 389aa517d744435bb043ff8c52d86c2d with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:26.304356 19107 ts_tablet_manager.cc:933] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:26.309451 19107 tablet_copy_client.cc:323] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:26.326503 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 80bc05a330c64471aaa556af75bea031 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:26.327034 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 80bc05a330c64471aaa556af75bea031 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-80bc05a330c64471aaa556af75bea031
I20260709 12:16:26.333478 18385 tablet_copy_source_session.cc:215] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:26.336472 19107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80bc05a330c64471aaa556af75bea031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:26.346021 19044 raft_consensus.cc:1217] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:26.362246 19107 tablet_copy_client.cc:806] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:26.362895 19107 tablet_copy_client.cc:670] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:26.366583 19107 tablet_copy_client.cc:538] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:26.374182 19107 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:26.393733 19107 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:26.394539 19107 tablet_bootstrap.cc:492] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:26.395138 19107 ts_tablet_manager.cc:1403] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:16:26.397891 19107 raft_consensus.cc:359] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.398758 19107 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:26.399477 19107 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:26.401381 19107 ts_tablet_manager.cc:1434] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:26.403142 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-80bc05a330c64471aaa556af75bea031 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:26.403586 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-80bc05a330c64471aaa556af75bea031 on tablet 80bc05a330c64471aaa556af75bea031 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:26.492753 19044 raft_consensus.cc:1217] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:26.530055 19044 raft_consensus.cc:1217] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:26.760864 19044 raft_consensus.cc:1217] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:26.831394 19088 raft_consensus.cc:1064] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:26.834345 19088 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:26.840339 18440 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:26.841848 18367 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:26.840830 19044 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:26.843976 19088 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:26.844879 19113 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:26.845770 18926 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:26.868237 18926 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.871142 19044 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.873845 18367 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.887857 18440 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.888262 18197 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.928785 18292 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:26.934224 19044 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:26.935564 18926 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:26.937925 18367 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:26.941144 19113 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:26.944137 19088 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.950655 19044 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.953425 18367 raft_consensus.cc:2955] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.955649 19076 raft_consensus.cc:1064] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:26.957332 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6ab0a3f9b34149918a57aec5d20b4884 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:26.964680 19076 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:26.965252 18195 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:26.999111 18295 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.000396 19044 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.003010 19081 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:27.004902 19115 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.014483 18373 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.016356 19081 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:27.027299 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ab0a3f9b34149918a57aec5d20b4884 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
W20260709 12:16:27.028177 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50219.5us
I20260709 12:16:27.033265 19131 tablet_replica.cc:333] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:27.034399 19131 raft_consensus.cc:2243] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:27.035478 19081 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.046418 19131 raft_consensus.cc:2272] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:27.045960 18295 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.051748 19131 ts_tablet_manager.cc:1916] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:27.051905 19044 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.057967 19128 raft_consensus.cc:1064] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:27.060398 19128 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:27.068377 19131 ts_tablet_manager.cc:1929] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:27.069296 19131 log.cc:1199] T 6ab0a3f9b34149918a57aec5d20b4884 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/6ab0a3f9b34149918a57aec5d20b4884
I20260709 12:16:27.071058 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 6ab0a3f9b34149918a57aec5d20b4884 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:27.065637 18195 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.080695 18367 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.081722 18295 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.083292 19081 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.083969 19044 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.092698 18367 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.093899 18440 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:27.105691 19115 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.111634 19081 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.119988 18295 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.121296 19043 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.124205 19076 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.149287 19081 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.149036 19044 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.152928 19115 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.166234 19128 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.167753 18297 raft_consensus.cc:2955] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.180733 19043 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.192520 18195 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.191423 18198 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.204193 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 389aa517d744435bb043ff8c52d86c2d with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:27.205255 18297 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.215111 18373 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.225399 19138 tablet_replica.cc:333] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:27.226504 19138 raft_consensus.cc:2243] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:27.227283 19138 raft_consensus.cc:2272] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:27.238099 19138 ts_tablet_manager.cc:1916] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:27.239634 18441 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:27.249177 19043 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.250604 19135 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.252138 18367 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.253679 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 389aa517d744435bb043ff8c52d86c2d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:27.255455 19138 ts_tablet_manager.cc:1929] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:27.255904 19138 log.cc:1199] T 389aa517d744435bb043ff8c52d86c2d P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/389aa517d744435bb043ff8c52d86c2d
I20260709 12:16:27.257551 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 389aa517d744435bb043ff8c52d86c2d (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:27.260768 19128 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.264729 19076 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.268778 19043 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.275583 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d266f68528964b1981ffe1df49861b08 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:27.280890 18195 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.295183 18367 raft_consensus.cc:2955] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.299695 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet d266f68528964b1981ffe1df49861b08 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:27.310415 19139 tablet_replica.cc:333] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:27.312069 19139 raft_consensus.cc:2243] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:27.313077 19139 raft_consensus.cc:2272] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:27.316637 19139 ts_tablet_manager.cc:1916] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:27.330231 19139 ts_tablet_manager.cc:1929] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:27.330710 19139 log.cc:1199] T d266f68528964b1981ffe1df49861b08 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/d266f68528964b1981ffe1df49861b08
I20260709 12:16:27.332593 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet d266f68528964b1981ffe1df49861b08 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:27.349130 18939 raft_consensus.cc:1064] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:27.351023 18939 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:27.358117 19043 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.358285 18441 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.359786 18939 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.360143 18295 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:27.362120 19116 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.363811 19116 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.383476 18920 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.385337 18295 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.396174 18441 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.404386 18195 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.394208 19043 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.417441 18367 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:27.427641 18295 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.429678 19043 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:27.432055 19116 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.434933 19116 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:27.440914 18939 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.444198 18295 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.450150 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 80bc05a330c64471aaa556af75bea031 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:27.450728 19043 raft_consensus.cc:2955] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:27.459162 18196 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.475528 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 80bc05a330c64471aaa556af75bea031 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:27.476423 19131 tablet_replica.cc:333] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:27.484427 19131 raft_consensus.cc:2243] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:27.485028 19131 raft_consensus.cc:2272] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:27.488379 19131 ts_tablet_manager.cc:1916] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:27.499336 19131 ts_tablet_manager.cc:1929] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:27.499717 19131 log.cc:1199] T 80bc05a330c64471aaa556af75bea031 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/80bc05a330c64471aaa556af75bea031
I20260709 12:16:27.501124 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 80bc05a330c64471aaa556af75bea031 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:28.543169 18367 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:16:28.549571 18338 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35973fcee55042b284c6f62e2681c304. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:28.550338 18440 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.551183 18294 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.551517 18939 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.552403 19145 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.560360 19145 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.568632 18297 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.575958 18440 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:28.578074 18198 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:28.588092 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80f93a4f5d0e45dd8b049fe76d3a0fda. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:28.590234 18295 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.589288 18440 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.591619 18367 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.594046 19081 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.595525 19150 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.628742 19150 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.633971 18294 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.652880 18367 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.656333 18373 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:28.668136 18294 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:28.668391 18338 consensus_peers.cc:597] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0931a54c049145dfad672173c0b29a40. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:28.669273 18440 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.669605 19116 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.672851 19116 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.677397 18198 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.702566 18440 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.703976 19145 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.716516 18295 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.717296 18196 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.727970 18295 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:28.742787 18440 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:28.744216 18263 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad852e5220a14f4898d2f069a5fb4a63. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:28.744475 18373 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:28.744781 19127 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.747094 19127 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:28.769182 19164 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.773599 18441 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.779817 18371 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.796926 18198 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:28.927500 19166 ts_tablet_manager.cc:933] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:28.930555 19166 tablet_copy_client.cc:323] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:28.932101 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 35973fcee55042b284c6f62e2681c304 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:28.932547 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 35973fcee55042b284c6f62e2681c304 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-35973fcee55042b284c6f62e2681c304
I20260709 12:16:28.938060 18385 tablet_copy_source_session.cc:215] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:28.942148 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35973fcee55042b284c6f62e2681c304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:28.955832 19166 tablet_copy_client.cc:806] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:28.956364 19166 tablet_copy_client.cc:670] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:28.959861 19166 tablet_copy_client.cc:538] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:28.968312 19166 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:28.987342 19166 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:28.988023 19166 tablet_bootstrap.cc:492] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:28.988552 19166 ts_tablet_manager.cc:1403] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 12:16:28.991046 19166 raft_consensus.cc:359] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:28.991882 19166 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:28.992475 19166 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:28.995296 19166 ts_tablet_manager.cc:1434] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.007s
I20260709 12:16:28.997170 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-35973fcee55042b284c6f62e2681c304 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:28.997620 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-35973fcee55042b284c6f62e2681c304 on tablet 35973fcee55042b284c6f62e2681c304 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:29.015075 19166 ts_tablet_manager.cc:933] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:29.016700 19166 tablet_copy_client.cc:323] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:29.018250 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:29.018683 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 80f93a4f5d0e45dd8b049fe76d3a0fda from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-80f93a4f5d0e45dd8b049fe76d3a0fda
I20260709 12:16:29.024544 18459 tablet_copy_source_session.cc:215] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:29.027400 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:29.041539 19166 tablet_copy_client.cc:806] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:29.042940 19166 tablet_copy_client.cc:670] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:29.046324 19166 tablet_copy_client.cc:538] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:29.054533 19166 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:29.077965 19166 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:29.078684 19166 tablet_bootstrap.cc:492] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:29.079214 19166 ts_tablet_manager.cc:1403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 12:16:29.081771 19166 raft_consensus.cc:359] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.082492 19166 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:29.083141 19166 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.085728 19166 ts_tablet_manager.cc:1434] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:16:29.087996 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-80f93a4f5d0e45dd8b049fe76d3a0fda received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:29.088390 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-80f93a4f5d0e45dd8b049fe76d3a0fda on tablet 80f93a4f5d0e45dd8b049fe76d3a0fda with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:29.184338 19166 ts_tablet_manager.cc:933] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:29.186770 19166 tablet_copy_client.cc:323] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:29.188228 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 0931a54c049145dfad672173c0b29a40 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:29.188751 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 0931a54c049145dfad672173c0b29a40 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-0931a54c049145dfad672173c0b29a40
I20260709 12:16:29.194406 18385 tablet_copy_source_session.cc:215] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:29.197398 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0931a54c049145dfad672173c0b29a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:29.211536 19166 tablet_copy_client.cc:806] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:29.212231 19166 tablet_copy_client.cc:670] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:29.226068 19166 tablet_copy_client.cc:538] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:29.232761 19166 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:29.246891 19166 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:29.247781 19166 tablet_bootstrap.cc:492] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:29.248983 19166 ts_tablet_manager.cc:1403] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.001s
I20260709 12:16:29.252075 19166 raft_consensus.cc:359] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.252620 19166 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:29.253135 19166 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.254606 19166 ts_tablet_manager.cc:1434] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:16:29.256330 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0931a54c049145dfad672173c0b29a40 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:29.256683 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0931a54c049145dfad672173c0b29a40 on tablet 0931a54c049145dfad672173c0b29a40 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:29.262809 19166 ts_tablet_manager.cc:933] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:29.267444 19166 tablet_copy_client.cc:323] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:29.268978 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet ad852e5220a14f4898d2f069a5fb4a63 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:29.269469 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet ad852e5220a14f4898d2f069a5fb4a63 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-ad852e5220a14f4898d2f069a5fb4a63
I20260709 12:16:29.275177 18310 tablet_copy_source_session.cc:215] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:29.277534 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad852e5220a14f4898d2f069a5fb4a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:29.286962 19166 tablet_copy_client.cc:806] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:29.287524 19166 tablet_copy_client.cc:670] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:29.293627 19166 tablet_copy_client.cc:538] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:29.298746 19166 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:29.314445 19166 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:29.315162 19166 tablet_bootstrap.cc:492] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:29.315711 19166 ts_tablet_manager.cc:1403] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.017s	user 0.018s	sys 0.000s
I20260709 12:16:29.317525 19166 raft_consensus.cc:359] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.318089 19166 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:29.318526 19166 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:29.320149 19166 ts_tablet_manager.cc:1434] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:16:29.322710 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ad852e5220a14f4898d2f069a5fb4a63 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:29.323081 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ad852e5220a14f4898d2f069a5fb4a63 on tablet ad852e5220a14f4898d2f069a5fb4a63 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:29.348816 19043 raft_consensus.cc:1217] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:29.509773 19043 raft_consensus.cc:1217] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:29.648187 19043 raft_consensus.cc:1217] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:29.803309 19043 raft_consensus.cc:1217] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:29.896196 19145 raft_consensus.cc:1064] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:29.898109 19145 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:29.904026 19043 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:29.904062 18446 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:29.905390 19160 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.905676 18294 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:29.907123 19160 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.908149 19145 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.931666 19160 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:29.944860 18295 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:29.945024 18446 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:29.947917 19043 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:29.974241 18195 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.008065 18371 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.017727 19043 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.018229 18446 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.019169 19160 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.020357 19081 raft_consensus.cc:1064] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:30.020961 19160 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.022538 19081 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.028716 18939 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.033968 19043 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.044834 18441 raft_consensus.cc:2955] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.056284 18196 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.068456 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 35973fcee55042b284c6f62e2681c304 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:30.072021 19190 tablet_replica.cc:333] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:30.074385 19190 raft_consensus.cc:2243] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.075404 19190 raft_consensus.cc:2272] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.080626 19190 ts_tablet_manager.cc:1916] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:30.098227 19190 ts_tablet_manager.cc:1929] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:30.098735 19190 log.cc:1199] T 35973fcee55042b284c6f62e2681c304 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/35973fcee55042b284c6f62e2681c304
I20260709 12:16:30.100468 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 35973fcee55042b284c6f62e2681c304 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:30.114081 19164 raft_consensus.cc:1064] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:30.123544 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35973fcee55042b284c6f62e2681c304 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:30.130846 19164 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.151599 19043 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.153214 19176 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.153523 18371 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.154830 19188 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.157224 18294 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.161687 19176 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.189914 19160 raft_consensus.cc:1064] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:30.190218 18441 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:30.192309 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:30.193488 19160 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.200018 19196 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.214412 19044 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.215903 19196 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.235975 18367 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.250681 19164 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.274197 19081 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.276224 19196 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.278656 18373 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.290398 18446 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.292691 19043 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.295945 18196 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:30.296401 18198 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:30.304086 18295 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.311096 19044 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.320125 18371 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.326215 19043 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.328666 18294 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.333120 19179 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.334707 19191 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.348337 18446 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:30.353492 19191 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:30.366902 18446 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.371799 18294 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.378762 19043 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.380318 19187 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.381806 18295 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.384614 19044 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.384788 19187 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.386236 19165 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.389233 19179 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.389976 19176 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.397675 19042 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.403432 18443 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.407456 19165 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.409610 19196 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:30.418560 18294 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.411661 19044 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.421911 19043 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.433807 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:30.436292 18446 raft_consensus.cc:2955] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.438565 18297 raft_consensus.cc:2955] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.445660 18196 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:30.439932 18443 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.463004 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 80f93a4f5d0e45dd8b049fe76d3a0fda with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:30.479840 18198 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.479720 18196 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:30.509711 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad852e5220a14f4898d2f069a5fb4a63 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:30.570261 19206 tablet_replica.cc:333] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:30.574350 18371 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:30.574841 19206 raft_consensus.cc:2243] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.575707 19206 raft_consensus.cc:2272] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.579214 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet ad852e5220a14f4898d2f069a5fb4a63 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:30.595613 19044 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.605257 18446 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:30.599805 19191 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:16:30.608831 19160 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:30.621413 18939 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.623245 19206 ts_tablet_manager.cc:1916] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:30.623477 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 57555.9us
I20260709 12:16:30.628147 18440 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.635124 19044 raft_consensus.cc:2955] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:30.643478 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0931a54c049145dfad672173c0b29a40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:30.650724 18195 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:30.664615 19206 ts_tablet_manager.cc:1929] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:30.666040 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 0931a54c049145dfad672173c0b29a40 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:30.665352 19206 log.cc:1199] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/80f93a4f5d0e45dd8b049fe76d3a0fda
I20260709 12:16:30.673755 19206 tablet_replica.cc:333] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:30.673632 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 80f93a4f5d0e45dd8b049fe76d3a0fda (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:30.674799 19206 raft_consensus.cc:2243] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.674919 19212 tablet_replica.cc:333] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:30.675787 19206 raft_consensus.cc:2272] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.676366 19212 raft_consensus.cc:2243] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.677214 19212 raft_consensus.cc:2272] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.679212 19206 ts_tablet_manager.cc:1916] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:30.681167 19212 ts_tablet_manager.cc:1916] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:30.694113 19206 ts_tablet_manager.cc:1929] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:30.694613 19206 log.cc:1199] T ad852e5220a14f4898d2f069a5fb4a63 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/ad852e5220a14f4898d2f069a5fb4a63
I20260709 12:16:30.695612 19212 ts_tablet_manager.cc:1929] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:30.696094 19212 log.cc:1199] T 0931a54c049145dfad672173c0b29a40 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/0931a54c049145dfad672173c0b29a40
I20260709 12:16:30.696434 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet ad852e5220a14f4898d2f069a5fb4a63 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:30.697818 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 0931a54c049145dfad672173c0b29a40 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:31.732925 18295 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:31.739115 18373 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:31.739190 18263 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d4d1763576e4c49849611360f37665a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.739777 18441 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.740453 19214 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:31.741261 19210 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:31.750535 19210 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.752444 18446 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.752767 18373 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.767824 18197 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:31.768666 18371 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:16:31.792361 18338 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5461f19f144642beb790dcf7f1515a2d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.797302 18446 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.802546 18295 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.799118 19191 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:16:31.809775 19179 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:31.827978 19145 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.829486 18295 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.830895 18446 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.840960 18441 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:31.848097 18371 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:31.849270 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3153d259c2254b2d9bdc71d8322cafc0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.849290 19176 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:31.854422 18295 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.854610 18196 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:31.855794 19187 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:31.868359 18371 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.866437 19187 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.882153 18295 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.887806 18373 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:31.904016 18195 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.922927 18441 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.923338 18297 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:31.926321 19191 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:31.927404 18338 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 548b719daf794fb881c5606ebec2abb3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.930625 19145 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:31.946853 19145 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.948390 18446 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.950295 18297 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:31.967212 18197 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:32.135187 19233 ts_tablet_manager.cc:933] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:32.137526 19233 tablet_copy_client.cc:323] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:32.139068 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 5d4d1763576e4c49849611360f37665a from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:32.139645 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 5d4d1763576e4c49849611360f37665a from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-5d4d1763576e4c49849611360f37665a
I20260709 12:16:32.146768 18310 tablet_copy_source_session.cc:215] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:32.149242 19233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4d1763576e4c49849611360f37665a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.160809 19233 tablet_copy_client.cc:806] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:32.161355 19233 tablet_copy_client.cc:670] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:32.165066 19233 tablet_copy_client.cc:538] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:32.172969 19233 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:32.192601 19233 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:32.193603 19233 tablet_bootstrap.cc:492] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:32.194406 19233 ts_tablet_manager.cc:1403] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.009s
I20260709 12:16:32.197326 19233 raft_consensus.cc:359] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.197995 19233 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:32.198542 19233 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.215164 19233 ts_tablet_manager.cc:1434] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.012s
I20260709 12:16:32.216830 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-5d4d1763576e4c49849611360f37665a received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:32.217258 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-5d4d1763576e4c49849611360f37665a on tablet 5d4d1763576e4c49849611360f37665a with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:32.227802 19233 ts_tablet_manager.cc:933] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:32.230373 19233 tablet_copy_client.cc:323] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:32.231954 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 5461f19f144642beb790dcf7f1515a2d from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:32.232458 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 5461f19f144642beb790dcf7f1515a2d from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-5461f19f144642beb790dcf7f1515a2d
I20260709 12:16:32.238308 18385 tablet_copy_source_session.cc:215] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:32.241099 19233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5461f19f144642beb790dcf7f1515a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.255645 19233 tablet_copy_client.cc:806] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:32.256187 19233 tablet_copy_client.cc:670] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:32.259150 19233 tablet_copy_client.cc:538] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:32.267351 19233 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:32.286993 19233 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:32.288319 19233 tablet_bootstrap.cc:492] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:32.288952 19233 ts_tablet_manager.cc:1403] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.003s
I20260709 12:16:32.291698 19233 raft_consensus.cc:359] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.292533 19233 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:32.293152 19233 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.295514 19233 ts_tablet_manager.cc:1434] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:32.297088 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-5461f19f144642beb790dcf7f1515a2d received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:32.297489 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-5461f19f144642beb790dcf7f1515a2d on tablet 5461f19f144642beb790dcf7f1515a2d with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:32.317639 19233 ts_tablet_manager.cc:933] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:32.318503 19239 ts_tablet_manager.cc:933] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:32.319640 19233 tablet_copy_client.cc:323] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:32.320561 19239 tablet_copy_client.cc:323] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:32.321343 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 3153d259c2254b2d9bdc71d8322cafc0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:32.321830 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 3153d259c2254b2d9bdc71d8322cafc0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-3153d259c2254b2d9bdc71d8322cafc0
I20260709 12:16:32.322217 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 548b719daf794fb881c5606ebec2abb3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:32.322759 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 548b719daf794fb881c5606ebec2abb3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-548b719daf794fb881c5606ebec2abb3
I20260709 12:16:32.329164 18459 tablet_copy_source_session.cc:215] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:32.330947 18385 tablet_copy_source_session.cc:215] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:32.332862 19233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3153d259c2254b2d9bdc71d8322cafc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.349442 19239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 548b719daf794fb881c5606ebec2abb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.360702 19233 tablet_copy_client.cc:806] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:32.361310 19233 tablet_copy_client.cc:670] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:32.370306 19233 tablet_copy_client.cc:538] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:32.377964 19233 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:32.378250 19239 tablet_copy_client.cc:806] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:32.378818 19239 tablet_copy_client.cc:670] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:32.382517 19239 tablet_copy_client.cc:538] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:32.393201 19239 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:32.401818 19233 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:32.402731 19233 tablet_bootstrap.cc:492] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:32.403815 19233 ts_tablet_manager.cc:1403] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.002s
I20260709 12:16:32.406407 19233 raft_consensus.cc:359] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.407150 19233 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:32.407740 19233 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.410029 19233 ts_tablet_manager.cc:1434] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:32.412668 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-3153d259c2254b2d9bdc71d8322cafc0 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:32.413071 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-3153d259c2254b2d9bdc71d8322cafc0 on tablet 3153d259c2254b2d9bdc71d8322cafc0 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:32.419369 19239 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:32.420290 19239 tablet_bootstrap.cc:492] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:32.420946 19239 ts_tablet_manager.cc:1403] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.004s
I20260709 12:16:32.424302 19239 raft_consensus.cc:359] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.425118 19239 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:32.425743 19239 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:32.428529 19239 ts_tablet_manager.cc:1434] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:16:32.430234 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-548b719daf794fb881c5606ebec2abb3 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:32.430657 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-548b719daf794fb881c5606ebec2abb3 on tablet 548b719daf794fb881c5606ebec2abb3 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:32.684347 19044 raft_consensus.cc:1217] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:32.779260 19044 raft_consensus.cc:1217] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:32.797176 19044 raft_consensus.cc:1217] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:32.937919 19044 raft_consensus.cc:1217] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:33.229759 19245 raft_consensus.cc:1064] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:33.233173 19245 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.243016 19044 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.245342 19228 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:33.248582 18371 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.249825 19228 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.256306 18295 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.257795 19228 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.261730 19228 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.268121 18371 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.297640 19247 raft_consensus.cc:1064] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:33.310686 18196 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.316346 18291 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.308454 19043 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.312101 19247 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.402575 19249 raft_consensus.cc:1064] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:33.405946 19145 raft_consensus.cc:1064] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:33.405866 19249 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.423192 19145 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.429674 18446 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.466480 19210 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:16:33.476203 19044 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.478035 19246 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:33.479498 18291 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.487318 19191 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.493501 18440 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.496512 19191 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.513319 19191 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.518894 18446 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.520533 19246 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.520529 18373 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.524549 19042 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.528223 19041 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.528973 19044 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.531809 19259 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:33.523102 19209 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:16:33.543761 18440 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.553684 18291 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:33.551506 19214 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:16:33.556528 18295 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.566442 19258 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.576515 19145 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:33.581276 19210 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.583545 18371 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.596947 19044 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.608949 18440 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.620795 18295 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.626238 19043 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.631822 18444 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.679334 18198 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.681499 18198 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.723981 18197 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.729663 18371 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.730701 18373 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.738306 19043 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.743963 18295 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.748258 18440 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.750869 19258 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:33.758558 18444 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:33.761194 19259 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.767151 19044 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.767483 18295 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.769019 19245 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.771530 19043 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.772948 18371 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.774278 19259 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.775115 19210 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.776043 19043 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.777136 19210 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.784613 18441 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.787204 19228 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.788128 19229 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:33.789011 19191 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.794924 19247 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.796188 18373 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.799834 19044 raft_consensus.cc:2955] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.801403 19246 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.806330 19042 raft_consensus.cc:2955] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.813652 18297 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.814284 19043 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.818766 18295 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:33.820967 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 548b719daf794fb881c5606ebec2abb3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:33.835858 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5461f19f144642beb790dcf7f1515a2d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:33.838092 19249 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:33.839596 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3153d259c2254b2d9bdc71d8322cafc0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:33.847613 18195 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.850063 18195 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.852536 18295 raft_consensus.cc:2955] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.853825 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5d4d1763576e4c49849611360f37665a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:33.853426 18198 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.857474 19041 raft_consensus.cc:2955] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.882949 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 3153d259c2254b2d9bdc71d8322cafc0 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:33.886781 19276 tablet_replica.cc:333] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:33.892598 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 5d4d1763576e4c49849611360f37665a with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:33.893695 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 5461f19f144642beb790dcf7f1515a2d with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:33.906981 18197 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:33.914237 19278 tablet_replica.cc:333] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:33.915390 19278 raft_consensus.cc:2243] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.916115 19278 raft_consensus.cc:2272] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.917413 19276 raft_consensus.cc:2243] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.918246 19276 raft_consensus.cc:2272] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.922082 19276 ts_tablet_manager.cc:1916] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:33.924463 19278 ts_tablet_manager.cc:1916] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:33.934387 19277 tablet_replica.cc:333] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:33.936533 19277 raft_consensus.cc:2243] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.938524 19277 raft_consensus.cc:2272] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.940831 19276 ts_tablet_manager.cc:1929] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:33.941378 19276 log.cc:1199] T 3153d259c2254b2d9bdc71d8322cafc0 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/3153d259c2254b2d9bdc71d8322cafc0
I20260709 12:16:33.945155 19277 ts_tablet_manager.cc:1916] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:33.951007 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 548b719daf794fb881c5606ebec2abb3 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:33.951082 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 3153d259c2254b2d9bdc71d8322cafc0 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:33.956426 19278 ts_tablet_manager.cc:1929] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:33.956916 19278 log.cc:1199] T 5461f19f144642beb790dcf7f1515a2d P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/5461f19f144642beb790dcf7f1515a2d
I20260709 12:16:33.958778 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 5461f19f144642beb790dcf7f1515a2d (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:33.962002 19277 ts_tablet_manager.cc:1929] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:33.962466 19277 log.cc:1199] T 5d4d1763576e4c49849611360f37665a P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/5d4d1763576e4c49849611360f37665a
I20260709 12:16:33.964308 19277 tablet_replica.cc:333] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:33.965229 19277 raft_consensus.cc:2243] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:33.965948 19277 raft_consensus.cc:2272] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:33.966154 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 5d4d1763576e4c49849611360f37665a (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:33.970386 19277 ts_tablet_manager.cc:1916] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:33.983842 19277 ts_tablet_manager.cc:1929] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:33.984153 19277 log.cc:1199] T 548b719daf794fb881c5606ebec2abb3 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/548b719daf794fb881c5606ebec2abb3
I20260709 12:16:33.985360 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 548b719daf794fb881c5606ebec2abb3 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:35.018982 18371 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.027479 18441 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:35.028725 18295 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:35.028982 18338 consensus_peers.cc:597] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc49a9d4c95c46f38ca4a279c913204d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.030603 19258 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.032315 19258 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.053459 19279 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.054924 18291 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.059284 18444 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.070420 18441 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.074079 18195 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:35.083459 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7d0369b4aea4afb960bf859e9b7738c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.084601 18371 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:35.085806 18291 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:35.086586 19281 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.088301 19245 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.106058 19281 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.115926 18291 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.116163 18371 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.134757 18291 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.139956 18198 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.141861 18440 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:35.143394 19209 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.147140 18371 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:35.148156 18263 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7dd86332c91f4c8d8616b22b5daaf743. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.149762 19256 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.166857 19209 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.170506 18441 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.170868 18373 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.185318 18195 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.185681 18373 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.194023 18441 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:35.194515 18338 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c749b52c1c8c44b78cee98b4b4d2ef29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.195463 19246 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.197444 18291 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:35.199127 19258 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.204845 19246 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.208603 18441 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.212010 18295 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:35.228205 18196 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.494488 19300 ts_tablet_manager.cc:933] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:35.496559 19300 tablet_copy_client.cc:323] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:35.498870 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet dc49a9d4c95c46f38ca4a279c913204d from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:35.499439 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet dc49a9d4c95c46f38ca4a279c913204d from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-dc49a9d4c95c46f38ca4a279c913204d
I20260709 12:16:35.505084 18385 tablet_copy_source_session.cc:215] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:35.507921 19300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc49a9d4c95c46f38ca4a279c913204d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:35.521494 19300 tablet_copy_client.cc:806] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:35.522022 19300 tablet_copy_client.cc:670] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:35.525528 19300 tablet_copy_client.cc:538] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:35.532244 19300 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:35.556267 19300 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:35.557158 19300 tablet_bootstrap.cc:492] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:35.557839 19300 ts_tablet_manager.cc:1403] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.004s
I20260709 12:16:35.560279 19300 raft_consensus.cc:359] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.560897 19300 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:35.561425 19300 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.564656 19300 ts_tablet_manager.cc:1434] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:35.566376 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-dc49a9d4c95c46f38ca4a279c913204d received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:35.566815 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-dc49a9d4c95c46f38ca4a279c913204d on tablet dc49a9d4c95c46f38ca4a279c913204d with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:35.671594 19300 ts_tablet_manager.cc:933] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:35.685103 19300 tablet_copy_client.cc:323] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:35.687891 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet e7d0369b4aea4afb960bf859e9b7738c from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:35.688481 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet e7d0369b4aea4afb960bf859e9b7738c from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-e7d0369b4aea4afb960bf859e9b7738c
I20260709 12:16:35.701779 18459 tablet_copy_source_session.cc:215] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:35.706256 19306 ts_tablet_manager.cc:933] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:35.706892 19300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d0369b4aea4afb960bf859e9b7738c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:35.713264 19307 ts_tablet_manager.cc:933] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:35.722030 19306 tablet_copy_client.cc:323] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:35.723961 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet c749b52c1c8c44b78cee98b4b4d2ef29 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:35.724484 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet c749b52c1c8c44b78cee98b4b4d2ef29 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-c749b52c1c8c44b78cee98b4b4d2ef29
I20260709 12:16:35.736574 19307 tablet_copy_client.cc:323] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:35.738467 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 7dd86332c91f4c8d8616b22b5daaf743 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:35.739049 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 7dd86332c91f4c8d8616b22b5daaf743 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-7dd86332c91f4c8d8616b22b5daaf743
I20260709 12:16:35.740568 18385 tablet_copy_source_session.cc:215] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:35.743963 19306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c749b52c1c8c44b78cee98b4b4d2ef29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:35.748092 19300 tablet_copy_client.cc:806] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:35.748647 19300 tablet_copy_client.cc:670] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:35.755157 19300 tablet_copy_client.cc:538] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:35.747042 18310 tablet_copy_source_session.cc:215] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:35.761485 19307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd86332c91f4c8d8616b22b5daaf743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:35.767370 19306 tablet_copy_client.cc:806] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:35.768527 19306 tablet_copy_client.cc:670] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:35.773185 19300 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:35.775529 19306 tablet_copy_client.cc:538] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:35.787896 19307 tablet_copy_client.cc:806] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:35.790020 19306 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:35.790228 19307 tablet_copy_client.cc:670] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:35.795099 19307 tablet_copy_client.cc:538] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:35.801373 19300 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:35.802294 19300 tablet_bootstrap.cc:492] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:35.802983 19300 ts_tablet_manager.cc:1403] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.003s
I20260709 12:16:35.804854 19307 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:35.806808 19300 raft_consensus.cc:359] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.807699 19300 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:35.808349 19300 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.813325 19300 ts_tablet_manager.cc:1434] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:35.816972 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-e7d0369b4aea4afb960bf859e9b7738c received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:35.817379 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-e7d0369b4aea4afb960bf859e9b7738c on tablet e7d0369b4aea4afb960bf859e9b7738c with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:35.826172 19306 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:35.827394 19306 tablet_bootstrap.cc:492] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:35.828147 19306 ts_tablet_manager.cc:1403] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.038s	user 0.022s	sys 0.005s
I20260709 12:16:35.832087 19306 raft_consensus.cc:359] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.833068 19306 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:35.833768 19306 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.836644 19306 ts_tablet_manager.cc:1434] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:16:35.837291 19307 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:35.839412 19307 tablet_bootstrap.cc:492] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:35.838238 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-c749b52c1c8c44b78cee98b4b4d2ef29 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:35.840103 19307 ts_tablet_manager.cc:1403] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.013s
I20260709 12:16:35.840247 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-c749b52c1c8c44b78cee98b4b4d2ef29 on tablet c749b52c1c8c44b78cee98b4b4d2ef29 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:35.843600 19307 raft_consensus.cc:359] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.844466 19307 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:35.845182 19307 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:35.847585 19307 ts_tablet_manager.cc:1434] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:35.849164 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-7dd86332c91f4c8d8616b22b5daaf743 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:35.849684 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-7dd86332c91f4c8d8616b22b5daaf743 on tablet 7dd86332c91f4c8d8616b22b5daaf743 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:35.942974 19041 raft_consensus.cc:1217] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:36.088223 19043 raft_consensus.cc:1217] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:36.196151 19043 raft_consensus.cc:1217] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:36.208891 19041 raft_consensus.cc:1217] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:36.503731 19292 raft_consensus.cc:1064] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:36.506067 19292 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.518327 19041 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.520036 19209 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.521126 18446 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.522642 18373 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.529701 19315 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:36.555810 19258 raft_consensus.cc:1064] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:36.558507 19209 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:36.563791 19258 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.591552 19209 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.597653 18373 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.598783 19041 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.600833 19299 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.600728 18446 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.608031 19304 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:36.619333 18295 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.619020 18198 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.623854 19304 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.623517 19043 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.626180 18441 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.629783 19245 raft_consensus.cc:1064] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:36.635264 19245 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.659106 19043 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.661893 18295 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.662715 19281 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.664292 19308 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:36.666682 19258 raft_consensus.cc:1064] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:36.666203 19304 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.668926 19258 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.672236 18367 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.672907 18297 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.677050 19308 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.712006 19281 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.712023 19043 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:36.742063 19328 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:16:36.744390 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57430.2us
I20260709 12:16:36.753348 18441 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.758172 18295 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.769299 18446 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.770720 19338 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.774796 18367 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.776003 19043 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.781318 18291 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:36.782240 19333 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:36.789894 19337 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:36.820945 19043 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.834970 19258 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.847719 18295 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.855602 18446 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.863801 19041 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.866500 18297 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.870196 18198 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:36.872093 18198 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.878756 19041 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.882465 18367 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.882709 19316 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:36.883980 19318 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:36.891304 19209 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.892737 19041 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.896644 18373 raft_consensus.cc:2955] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.900219 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7dd86332c91f4c8d8616b22b5daaf743 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:36.915102 18441 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.915620 18194 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.918243 18194 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.923147 18373 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.926693 18297 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.931325 19042 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.932906 19041 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.934286 19344 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:36.943749 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dd86332c91f4c8d8616b22b5daaf743 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:36.944031 18446 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:36.945662 19341 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.947474 19043 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.957238 19330 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:36.959403 19337 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:36.960830 19245 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:36.962553 19348 tablet_replica.cc:333] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:36.967204 19348 raft_consensus.cc:2243] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:36.968022 19348 raft_consensus.cc:2272] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:36.959686 18195 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.974972 18446 raft_consensus.cc:2955] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.979439 19334 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.983352 19043 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.983764 18297 raft_consensus.cc:2955] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:36.987128 18367 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:36.991217 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e7d0369b4aea4afb960bf859e9b7738c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:36.995751 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc49a9d4c95c46f38ca4a279c913204d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:36.996274 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet dc49a9d4c95c46f38ca4a279c913204d with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:37.004750 19041 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:37.002494 18195 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:37.008180 18295 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:37.009469 19299 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:37.012761 19299 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:37.022588 18297 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:37.020984 19341 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:37.031066 19041 raft_consensus.cc:2955] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:37.038429 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet e7d0369b4aea4afb960bf859e9b7738c with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:37.038848 19348 ts_tablet_manager.cc:1916] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:37.043812 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c749b52c1c8c44b78cee98b4b4d2ef29 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:37.060626 19350 tablet_replica.cc:333] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:37.060774 19349 tablet_replica.cc:333] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:37.068822 19349 raft_consensus.cc:2243] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:37.069603 19349 raft_consensus.cc:2272] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:37.063875 18198 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:37.067576 19350 raft_consensus.cc:2243] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:37.074358 19350 raft_consensus.cc:2272] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:37.078228 19349 ts_tablet_manager.cc:1916] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:37.080318 19350 ts_tablet_manager.cc:1916] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:37.081808 19348 ts_tablet_manager.cc:1929] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:37.090512 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet c749b52c1c8c44b78cee98b4b4d2ef29 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:37.099056 19348 log.cc:1199] T 7dd86332c91f4c8d8616b22b5daaf743 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/7dd86332c91f4c8d8616b22b5daaf743
I20260709 12:16:37.100606 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 7dd86332c91f4c8d8616b22b5daaf743 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:37.101024 19348 tablet_replica.cc:333] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:37.103396 19348 raft_consensus.cc:2243] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:37.103960 19348 raft_consensus.cc:2272] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:37.107645 19348 ts_tablet_manager.cc:1916] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:37.112691 19350 ts_tablet_manager.cc:1929] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:37.113191 19350 log.cc:1199] T e7d0369b4aea4afb960bf859e9b7738c P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/e7d0369b4aea4afb960bf859e9b7738c
I20260709 12:16:37.113632 19349 ts_tablet_manager.cc:1929] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:37.115887 19349 log.cc:1199] T dc49a9d4c95c46f38ca4a279c913204d P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/dc49a9d4c95c46f38ca4a279c913204d
I20260709 12:16:37.116370 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet e7d0369b4aea4afb960bf859e9b7738c (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:37.117519 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet dc49a9d4c95c46f38ca4a279c913204d (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:37.122205 19348 ts_tablet_manager.cc:1929] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:37.122597 19348 log.cc:1199] T c749b52c1c8c44b78cee98b4b4d2ef29 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/c749b52c1c8c44b78cee98b4b4d2ef29
I20260709 12:16:37.123894 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet c749b52c1c8c44b78cee98b4b4d2ef29 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:38.165475 18446 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.170857 18367 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:38.170915 18295 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:38.171249 18412 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5398d6474dd4eed946f45af9214825c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.172272 19334 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.173110 19308 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.184818 19334 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.189781 18367 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.188403 18295 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.199054 18297 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.205729 18195 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.206903 18367 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:38.208545 18446 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:38.208525 19316 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.210064 19315 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:38.213500 18263 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a971a418238d45d288b49960c0abeea0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.229624 18371 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.225610 19316 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.240907 18444 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.258766 18198 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.259617 18297 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:16:38.269340 18263 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0071c33745ae42e48eca14d86aa5fac6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.274896 18444 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:38.279584 18371 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:38.278501 19363 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.281597 19315 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.289644 19315 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.291535 18371 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.297178 18444 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.304674 18297 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.1, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.309721 18198 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.321844 18446 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:16:38.322455 18263 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e98a95c8e3c14bcfa51b054f4829293b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.323853 18371 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:38.324138 19315 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.325019 19363 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:38.338920 19363 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.341681 18371 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.354550 18441 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Committing config change with OpId 3.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.361624 18198 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:38.659166 19372 ts_tablet_manager.cc:933] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:38.661226 19372 tablet_copy_client.cc:323] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:38.662601 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet c5398d6474dd4eed946f45af9214825c from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:38.663117 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet c5398d6474dd4eed946f45af9214825c from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-c5398d6474dd4eed946f45af9214825c
I20260709 12:16:38.672916 18459 tablet_copy_source_session.cc:215] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:38.678750 19372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5398d6474dd4eed946f45af9214825c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:38.693490 19372 tablet_copy_client.cc:806] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:38.693985 19372 tablet_copy_client.cc:670] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:38.697412 19372 tablet_copy_client.cc:538] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:38.704131 19372 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:38.722482 19372 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:38.723405 19372 tablet_bootstrap.cc:492] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:38.724086 19372 ts_tablet_manager.cc:1403] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.015s
I20260709 12:16:38.729259 19372 raft_consensus.cc:359] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.730280 19372 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:38.730932 19372 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.735805 19372 ts_tablet_manager.cc:1434] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:16:38.737538 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-c5398d6474dd4eed946f45af9214825c received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:38.737941 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-c5398d6474dd4eed946f45af9214825c on tablet c5398d6474dd4eed946f45af9214825c with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:38.749202 19372 ts_tablet_manager.cc:933] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:38.751569 19372 tablet_copy_client.cc:323] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:38.752993 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet e98a95c8e3c14bcfa51b054f4829293b from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:38.753484 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet e98a95c8e3c14bcfa51b054f4829293b from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-e98a95c8e3c14bcfa51b054f4829293b
I20260709 12:16:38.759967 19377 ts_tablet_manager.cc:933] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:38.760078 18310 tablet_copy_source_session.cc:215] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:38.765228 19372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98a95c8e3c14bcfa51b054f4829293b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:38.770144 19377 tablet_copy_client.cc:323] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:38.771996 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet a971a418238d45d288b49960c0abeea0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:38.772542 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet a971a418238d45d288b49960c0abeea0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-a971a418238d45d288b49960c0abeea0
I20260709 12:16:38.779853 18310 tablet_copy_source_session.cc:215] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:38.785873 19372 tablet_copy_client.cc:806] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:38.786753 19372 tablet_copy_client.cc:670] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:38.787635 19377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a971a418238d45d288b49960c0abeea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:38.796978 19372 tablet_copy_client.cc:538] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:38.807011 19372 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:38.811115 19377 tablet_copy_client.cc:806] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:38.811990 19377 tablet_copy_client.cc:670] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:38.816617 19377 tablet_copy_client.cc:538] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:38.827282 19377 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:38.829515 19372 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:38.830516 19372 tablet_bootstrap.cc:492] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:38.831211 19372 ts_tablet_manager.cc:1403] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.013s
I20260709 12:16:38.834631 19372 raft_consensus.cc:359] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.835597 19372 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:38.836445 19372 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.838824 19372 ts_tablet_manager.cc:1434] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:16:38.840555 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-e98a95c8e3c14bcfa51b054f4829293b received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:38.840960 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-e98a95c8e3c14bcfa51b054f4829293b on tablet e98a95c8e3c14bcfa51b054f4829293b with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:38.859885 19377 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:38.860805 19377 tablet_bootstrap.cc:492] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:38.861514 19377 ts_tablet_manager.cc:1403] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.006s
I20260709 12:16:38.862473 19372 ts_tablet_manager.cc:933] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:38.864117 19372 tablet_copy_client.cc:323] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:38.864336 19377 raft_consensus.cc:359] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.865180 19377 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:38.865880 19377 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.867216 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 0071c33745ae42e48eca14d86aa5fac6 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:38.867859 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 0071c33745ae42e48eca14d86aa5fac6 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-0071c33745ae42e48eca14d86aa5fac6
I20260709 12:16:38.868691 19377 ts_tablet_manager.cc:1434] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:38.870445 18309 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-a971a418238d45d288b49960c0abeea0 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:38.870868 18309 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-a971a418238d45d288b49960c0abeea0 on tablet a971a418238d45d288b49960c0abeea0 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:38.881564 18310 tablet_copy_source_session.cc:215] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:38.885042 19372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0071c33745ae42e48eca14d86aa5fac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:38.896854 19372 tablet_copy_client.cc:806] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:38.897410 19372 tablet_copy_client.cc:670] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:38.905170 19372 tablet_copy_client.cc:538] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:38.913354 19372 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:38.930214 19372 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:38.931083 19372 tablet_bootstrap.cc:492] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:38.931717 19372 ts_tablet_manager.cc:1403] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.011s
I20260709 12:16:38.934211 19372 raft_consensus.cc:359] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.934954 19372 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:38.935519 19372 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:38.937538 19372 ts_tablet_manager.cc:1434] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:38.939510 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0071c33745ae42e48eca14d86aa5fac6 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:38.939894 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0071c33745ae42e48eca14d86aa5fac6 on tablet 0071c33745ae42e48eca14d86aa5fac6 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:39.146872 19043 raft_consensus.cc:1217] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:39.286095 19043 raft_consensus.cc:1217] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:39.336774 19043 raft_consensus.cc:1217] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Deduplicated request from leader. Original: 3.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 12:16:39.383440 19043 raft_consensus.cc:1217] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:39.690573 19383 raft_consensus.cc:1064] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:39.692552 19383 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:39.699370 19041 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.700596 19385 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:39.703406 18291 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.704293 18367 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.705052 19387 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:39.706187 19383 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:39.729820 19385 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.732215 18295 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.755916 19043 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.756959 18371 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.763640 18196 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:39.784863 19384 raft_consensus.cc:1064] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:39.788156 19384 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:39.801589 18444 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.803609 18367 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.803830 19363 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:39.802129 19043 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:39.806882 19352 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:39.809113 19352 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:39.838338 19352 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.841213 18446 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.843850 18444 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:39.849884 19043 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:39.851140 18367 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:39.853003 19383 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:39.854125 19387 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:39.855944 18371 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.869719 19387 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.882169 18194 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.881835 18367 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:39.886575 19043 raft_consensus.cc:2955] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:39.941581 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72951.7us
I20260709 12:16:39.943588 19401 raft_consensus.cc:1064] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
W20260709 12:16:39.955832 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 88209us
W20260709 12:16:39.959403 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:39.966181 19384 raft_consensus.cc:1064] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:39.996290 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c5398d6474dd4eed946f45af9214825c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:39.986393 19384 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:39.990499 19401 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 0, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:40.010895 19043 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.044627 18194 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:40.049695 18295 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:16:40.055698 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 101598us
I20260709 12:16:40.082959 18446 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:40.090034 19363 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:40.096843 18446 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:40.099103 19403 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:40.099813 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet c5398d6474dd4eed946f45af9214825c with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:40.103324 19407 tablet_replica.cc:333] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:40.106999 18372 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:40.107486 19042 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:40.109304 19384 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:40.112586 18441 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:40.116238 19384 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:40.116688 19403 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.119074 19402 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:40.129686 18442 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.131711 19041 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:40.135905 19041 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:40.137494 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a971a418238d45d288b49960c0abeea0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:40.135593 19401 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:40.138072 19402 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:40.138200 18370 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:40.142843 19384 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:40.106995 19407 raft_consensus.cc:2243] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:40.146215 19041 raft_consensus.cc:2955] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.148093 19407 raft_consensus.cc:2272] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:40.157574 19407 ts_tablet_manager.cc:1916] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:40.161818 19384 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.163054 18198 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.167514 18442 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.173120 19363 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.179695 19041 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.181176 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet a971a418238d45d288b49960c0abeea0 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:40.184306 18372 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.187510 18444 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.189569 18370 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.204982 19044 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.211763 19407 ts_tablet_manager.cc:1929] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:40.212306 19407 log.cc:1199] T c5398d6474dd4eed946f45af9214825c P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/c5398d6474dd4eed946f45af9214825c
I20260709 12:16:40.215409 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet c5398d6474dd4eed946f45af9214825c (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:40.210788 18195 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.227888 19414 tablet_replica.cc:333] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:40.229012 19414 raft_consensus.cc:2243] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:40.229771 19414 raft_consensus.cc:2272] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:40.235085 19414 ts_tablet_manager.cc:1916] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:40.244318 18194 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.251344 19414 ts_tablet_manager.cc:1929] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:40.251864 19414 log.cc:1199] T a971a418238d45d288b49960c0abeea0 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/a971a418238d45d288b49960c0abeea0
I20260709 12:16:40.253815 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet a971a418238d45d288b49960c0abeea0 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:40.265767 18297 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:40.269769 18295 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:40.271790 18370 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:40.274881 18442 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:40.276317 19384 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:40.270130 19044 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:40.280366 19043 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:40.280278 19363 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:40.281733 19402 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:40.286495 19402 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:40.289686 19363 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.300302 19404 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.291105 19044 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.315481 19043 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.321046 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0071c33745ae42e48eca14d86aa5fac6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:40.333575 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e98a95c8e3c14bcfa51b054f4829293b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:40.322086 18442 raft_consensus.cc:2955] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.322916 18372 raft_consensus.cc:2955] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.339439 18198 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.351662 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 0071c33745ae42e48eca14d86aa5fac6 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:40.358479 19414 tablet_replica.cc:333] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:40.362115 19414 raft_consensus.cc:2243] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:40.362443 18197 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:40.379592 19414 raft_consensus.cc:2272] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:40.383183 19414 ts_tablet_manager.cc:1916] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:40.391891 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet e98a95c8e3c14bcfa51b054f4829293b with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:40.393730 19415 tablet_replica.cc:333] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:40.394711 19415 raft_consensus.cc:2243] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:40.395968 19415 raft_consensus.cc:2272] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:40.396898 19414 ts_tablet_manager.cc:1929] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:40.397352 19414 log.cc:1199] T 0071c33745ae42e48eca14d86aa5fac6 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/0071c33745ae42e48eca14d86aa5fac6
I20260709 12:16:40.398778 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 0071c33745ae42e48eca14d86aa5fac6 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:40.400183 19415 ts_tablet_manager.cc:1916] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:40.410567 19415 ts_tablet_manager.cc:1929] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.3
I20260709 12:16:40.410907 19415 log.cc:1199] T e98a95c8e3c14bcfa51b054f4829293b P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/e98a95c8e3c14bcfa51b054f4829293b
I20260709 12:16:40.412559 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet e98a95c8e3c14bcfa51b054f4829293b (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:41.464533 18371 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:41.470026 18297 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:41.470400 18444 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:41.470556 18338 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bc6be8433ea4f43a3c0ac6dfe27398e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.471423 19341 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.472218 19351 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.480808 19341 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.482913 18297 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.490478 18444 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.496178 18440 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:41.505609 18371 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:41.507091 19308 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.507292 18198 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.509986 18297 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:41.511281 18412 consensus_peers.cc:597] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37ef0235d410491ab47198988075f6e4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.512207 19308 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.516803 19387 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.518743 18371 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.540426 18295 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.541405 18297 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:41.554175 18440 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:41.557757 19363 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:41.559760 18371 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:41.561971 19403 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
W20260709 12:16:41.575054 18263 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e4622d18b474b659ebea19e9aad0d05. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.610780 19429 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.611210 18194 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.623430 18440 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.633642 18372 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.635213 18444 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:41.642328 18195 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:41.656514 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fafcfc4614304a81b0eadc8f445bfdb8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.657676 18371 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:41.659118 19387 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.662834 18297 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:41.667992 19387 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.694047 19427 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.701735 18297 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.713231 18371 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:41.719974 18194 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.895466 19437 ts_tablet_manager.cc:933] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:41.897516 19437 tablet_copy_client.cc:323] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:41.899029 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 37ef0235d410491ab47198988075f6e4 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:41.899605 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 37ef0235d410491ab47198988075f6e4 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-37ef0235d410491ab47198988075f6e4
I20260709 12:16:41.908422 18459 tablet_copy_source_session.cc:215] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:41.911110 19437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ef0235d410491ab47198988075f6e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:41.926446 19437 tablet_copy_client.cc:806] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:41.926988 19437 tablet_copy_client.cc:670] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:41.930873 19437 tablet_copy_client.cc:538] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:41.936596 19437 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:41.999131 19437 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:41.999935 19437 tablet_bootstrap.cc:492] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:42.000491 19437 ts_tablet_manager.cc:1403] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.064s	user 0.027s	sys 0.034s
I20260709 12:16:42.002839 19437 raft_consensus.cc:359] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.003482 19437 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:42.003958 19437 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.006645 19437 ts_tablet_manager.cc:1434] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:16:42.010246 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-37ef0235d410491ab47198988075f6e4 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:42.010676 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-37ef0235d410491ab47198988075f6e4 on tablet 37ef0235d410491ab47198988075f6e4 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:42.075867 19437 ts_tablet_manager.cc:933] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:42.079481 19437 tablet_copy_client.cc:323] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:42.081039 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:42.081564 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 8bc6be8433ea4f43a3c0ac6dfe27398e from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-8bc6be8433ea4f43a3c0ac6dfe27398e
I20260709 12:16:42.088814 18385 tablet_copy_source_session.cc:215] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:42.096037 19437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:42.112833 19437 tablet_copy_client.cc:806] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:42.113425 19437 tablet_copy_client.cc:670] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:42.116650 19437 tablet_copy_client.cc:538] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:42.124389 19437 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:42.140345 19443 ts_tablet_manager.cc:933] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:42.145011 19443 tablet_copy_client.cc:323] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:42.146579 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 2e4622d18b474b659ebea19e9aad0d05 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:42.147009 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 2e4622d18b474b659ebea19e9aad0d05 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-2e4622d18b474b659ebea19e9aad0d05
I20260709 12:16:42.149492 19437 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:42.150417 19437 tablet_bootstrap.cc:492] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:42.151077 19437 ts_tablet_manager.cc:1403] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.005s
I20260709 12:16:42.153503 18310 tablet_copy_source_session.cc:215] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:42.153992 19437 raft_consensus.cc:359] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.154873 19437 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:42.155550 19437 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.158571 19437 ts_tablet_manager.cc:1434] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:42.160508 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8bc6be8433ea4f43a3c0ac6dfe27398e received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:42.160899 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8bc6be8433ea4f43a3c0ac6dfe27398e on tablet 8bc6be8433ea4f43a3c0ac6dfe27398e with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:42.163424 19443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4622d18b474b659ebea19e9aad0d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:42.179040 19443 tablet_copy_client.cc:806] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:42.179569 19443 tablet_copy_client.cc:670] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:42.184119 19443 tablet_copy_client.cc:538] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:42.193051 19443 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:42.216310 19443 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:42.217235 19443 tablet_bootstrap.cc:492] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:42.217865 19443 ts_tablet_manager.cc:1403] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.001s
I20260709 12:16:42.220736 19443 raft_consensus.cc:359] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.221834 19443 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:42.222491 19443 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.225227 19443 ts_tablet_manager.cc:1434] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:42.230916 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-2e4622d18b474b659ebea19e9aad0d05 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:42.231308 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-2e4622d18b474b659ebea19e9aad0d05 on tablet 2e4622d18b474b659ebea19e9aad0d05 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:42.272220 19443 ts_tablet_manager.cc:933] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:42.280735 19443 tablet_copy_client.cc:323] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:42.295825 19043 raft_consensus.cc:1217] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:42.332754 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet fafcfc4614304a81b0eadc8f445bfdb8 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:42.333760 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet fafcfc4614304a81b0eadc8f445bfdb8 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-fafcfc4614304a81b0eadc8f445bfdb8
I20260709 12:16:42.350087 18459 tablet_copy_source_session.cc:215] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:42.391973 19443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fafcfc4614304a81b0eadc8f445bfdb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:42.440614 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 82985.4us
I20260709 12:16:42.465082 19443 tablet_copy_client.cc:806] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:42.466187 19443 tablet_copy_client.cc:670] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:42.473011 19443 tablet_copy_client.cc:538] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:42.485353 19443 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:42.550345 19443 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:42.551391 19443 tablet_bootstrap.cc:492] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:42.552239 19443 ts_tablet_manager.cc:1403] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.011s
I20260709 12:16:42.554800 19443 raft_consensus.cc:359] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.555751 19443 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:42.556571 19443 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:42.559240 19443 ts_tablet_manager.cc:1434] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:42.631368 19042 raft_consensus.cc:1217] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:16:42.673471 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52732.8us
I20260709 12:16:42.762372 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-fafcfc4614304a81b0eadc8f445bfdb8 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:42.762837 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-fafcfc4614304a81b0eadc8f445bfdb8 on tablet fafcfc4614304a81b0eadc8f445bfdb8 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:42.769011 19043 raft_consensus.cc:1217] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:16:42.771202 19037 raft_consensus.cc:1217] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:16:42.804816 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 151057us
I20260709 12:16:42.841429 19436 raft_consensus.cc:1064] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:42.856951 19436 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:42.874047 19039 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:42.875466 19448 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:42.878602 18294 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:42.880535 19308 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:42.884208 18367 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:42.885732 19466 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:42.900521 19448 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.902117 18367 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.906113 18294 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.917073 19039 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.918923 18197 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:42.949326 18440 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:42.955404 18367 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:42.956875 19308 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:42.961761 19039 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:42.963879 19466 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:42.964934 19308 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.968179 18367 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.970647 19039 raft_consensus.cc:2955] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.977351 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 37ef0235d410491ab47198988075f6e4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:42.985025 18195 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:42.994234 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 37ef0235d410491ab47198988075f6e4 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:42.996696 19475 tablet_replica.cc:333] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:42.997496 19475 raft_consensus.cc:2243] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.998145 19475 raft_consensus.cc:2272] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:43.002138 19475 ts_tablet_manager.cc:1916] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:43.018194 19475 ts_tablet_manager.cc:1929] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:43.018502 19475 log.cc:1199] T 37ef0235d410491ab47198988075f6e4 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/37ef0235d410491ab47198988075f6e4
I20260709 12:16:43.019861 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 37ef0235d410491ab47198988075f6e4 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:43.041952 19431 raft_consensus.cc:1064] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:43.044229 19431 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.050555 18445 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:43.050698 19039 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:43.052294 19363 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:43.054344 19452 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:43.054649 18367 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:43.056097 19363 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:43.075367 19363 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.078466 19039 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.100242 18367 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.104291 18445 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.108457 18194 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:43.138204 18298 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.147112 19037 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:43.148072 18369 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:43.150535 19419 raft_consensus.cc:1064] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:43.150359 19455 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:43.156678 19431 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:43.157501 19419 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.169001 19429 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.173257 19044 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.176658 18445 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.175855 18369 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.182225 19465 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.191746 19458 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.220428 19037 raft_consensus.cc:2955] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.230245 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2e4622d18b474b659ebea19e9aad0d05 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:43.237965 18197 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 4 to 5, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.238590 19460 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.245659 18294 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.240264 19039 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.250540 19458 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:16:43.254696 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e4622d18b474b659ebea19e9aad0d05 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:43.262404 19466 raft_consensus.cc:1064] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:43.264394 19485 tablet_replica.cc:333] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:43.264920 19466 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.265620 19485 raft_consensus.cc:2243] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:43.266387 19485 raft_consensus.cc:2272] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:43.271428 19485 ts_tablet_manager.cc:1916] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:43.270988 18445 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.279459 18197 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.288712 19485 ts_tablet_manager.cc:1929] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:16:43.289682 19485 log.cc:1199] T 2e4622d18b474b659ebea19e9aad0d05 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/2e4622d18b474b659ebea19e9aad0d05
I20260709 12:16:43.301824 18374 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.304040 18300 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.304239 19427 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.306957 19436 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.311725 18294 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.318199 19039 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:43.320627 19448 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.344166 19308 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.382956 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 2e4622d18b474b659ebea19e9aad0d05 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:43.406920 18374 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.405064 18369 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.397321 18298 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.426885 19044 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.444538 18440 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:43.447518 19460 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:43.447698 18197 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:43.451390 19038 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:43.464947 19457 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:43.500447 18445 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:43.520107 19457 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.526021 18440 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.543660 19044 raft_consensus.cc:2955] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.552780 19039 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:43.563122 19308 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:43.573762 18197 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.578675 18298 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:43.575443 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:43.583484 19427 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:43.585740 19427 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.589429 19039 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.607493 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bc6be8433ea4f43a3c0ac6dfe27398e with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:43.605273 18294 raft_consensus.cc:2955] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.618930 19492 tablet_replica.cc:333] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:43.640872 19492 raft_consensus.cc:2243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:43.641738 19492 raft_consensus.cc:2272] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:43.645496 18197 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:43.647845 19492 ts_tablet_manager.cc:1916] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:43.657063 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet fafcfc4614304a81b0eadc8f445bfdb8 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:43.658176 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fafcfc4614304a81b0eadc8f445bfdb8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:43.676146 19493 tablet_replica.cc:333] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:43.678045 19493 raft_consensus.cc:2243] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:43.678843 19493 raft_consensus.cc:2272] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:43.682966 19493 ts_tablet_manager.cc:1916] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:43.688959 19492 ts_tablet_manager.cc:1929] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:43.689607 19492 log.cc:1199] T 8bc6be8433ea4f43a3c0ac6dfe27398e P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/8bc6be8433ea4f43a3c0ac6dfe27398e
I20260709 12:16:43.691437 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 8bc6be8433ea4f43a3c0ac6dfe27398e (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:43.699110 19493 ts_tablet_manager.cc:1929] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:43.699568 19493 log.cc:1199] T fafcfc4614304a81b0eadc8f445bfdb8 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/fafcfc4614304a81b0eadc8f445bfdb8
I20260709 12:16:43.701005 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet fafcfc4614304a81b0eadc8f445bfdb8 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
W20260709 12:16:43.746591 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002e24a0 after lost signal to thread 17160
W20260709 12:16:43.747669 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08001a0980 after lost signal to thread 18232
W20260709 12:16:43.748831 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800339960 after lost signal to thread 18324
W20260709 12:16:43.749965 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002ca680 after lost signal to thread 18399
W20260709 12:16:43.750998 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800211720 after lost signal to thread 18473
W20260709 12:16:43.752290 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800339700 after lost signal to thread 19068
W20260709 12:16:43.752832 19065 debug-util.cc:398] Leaking SignalData structure 0x7b080030e6a0 after lost signal to thread 19433
I20260709 12:16:44.763723 18297 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:44.769747 18447 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.771476 18375 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.771334 19495 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:44.773288 19495 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:44.776139 18263 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9f2e3c205c14e43b532c52bc7fa4c90. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:44.783288 19452 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.784984 18375 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.788820 18447 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.801116 18369 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:44.805148 18197 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:44.822839 18338 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b135a527cce4b14b789d23618d411b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:44.813537 18445 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.832464 19498 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:44.844027 18294 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.854135 19464 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:44.864511 19464 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.868265 18446 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.877619 18291 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.920819 18291 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:44.921733 18194 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:16:44.944947 18263 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0936d54d511a4cc3903264974a56ecb7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:44.945462 18446 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.945875 18369 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:44.946724 19502 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:44.947657 19495 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:44.963608 19495 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.965267 18447 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:44.990206 18194 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:45.001075 18369 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.001278 18375 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:45.029112 18291 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:45.030694 19497 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:45.031693 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adea31c273544d04b7f16f9b9a3ae610. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.030674 18447 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:45.039907 19497 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:45.054302 19498 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:45.058971 18447 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:45.058971 18291 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:45.097999 18197 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:16:45.249928 18321 debug-util.cc:398] Leaking SignalData structure 0x7b08001d77a0 after lost signal to thread 17160
W20260709 12:16:45.250931 18321 debug-util.cc:398] Leaking SignalData structure 0x7b08001bd5c0 after lost signal to thread 18232
W20260709 12:16:45.252081 18321 debug-util.cc:398] Leaking SignalData structure 0x7b080032c480 after lost signal to thread 18324
W20260709 12:16:45.253080 18321 debug-util.cc:398] Leaking SignalData structure 0x7b080010f020 after lost signal to thread 18399
W20260709 12:16:45.254074 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800338040 after lost signal to thread 18473
W20260709 12:16:45.255096 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800108f60 after lost signal to thread 19068
I20260709 12:16:45.296393 19520 ts_tablet_manager.cc:933] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:45.305258 19520 tablet_copy_client.cc:323] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:45.307614 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet b9f2e3c205c14e43b532c52bc7fa4c90 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:45.308161 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet b9f2e3c205c14e43b532c52bc7fa4c90 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-b9f2e3c205c14e43b532c52bc7fa4c90
I20260709 12:16:45.317943 18310 tablet_copy_source_session.cc:215] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:45.321983 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f2e3c205c14e43b532c52bc7fa4c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.353264 19520 tablet_copy_client.cc:806] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:45.353971 19520 tablet_copy_client.cc:670] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:45.359207 19520 tablet_copy_client.cc:538] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:45.368924 19520 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:45.404496 19522 ts_tablet_manager.cc:933] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:45.421867 19520 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:45.422957 19520 tablet_bootstrap.cc:492] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:45.423736 19520 ts_tablet_manager.cc:1403] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.010s
I20260709 12:16:45.427037 19520 raft_consensus.cc:359] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.432938 19520 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:45.433750 19520 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.438766 19522 tablet_copy_client.cc:323] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:45.440990 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 4b135a527cce4b14b789d23618d411b0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:45.441596 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 4b135a527cce4b14b789d23618d411b0 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-4b135a527cce4b14b789d23618d411b0
I20260709 12:16:45.450443 18385 tablet_copy_source_session.cc:215] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:45.459550 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b135a527cce4b14b789d23618d411b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.463719 19520 ts_tablet_manager.cc:1434] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.005s
I20260709 12:16:45.467604 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-b9f2e3c205c14e43b532c52bc7fa4c90 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:45.468107 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-b9f2e3c205c14e43b532c52bc7fa4c90 on tablet b9f2e3c205c14e43b532c52bc7fa4c90 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.505568 19522 tablet_copy_client.cc:806] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:45.507083 19522 tablet_copy_client.cc:670] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:45.512276 19522 tablet_copy_client.cc:538] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:45.539762 19520 ts_tablet_manager.cc:933] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:45.554105 19522 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:45.556726 19520 tablet_copy_client.cc:323] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:45.560436 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 0936d54d511a4cc3903264974a56ecb7 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:45.560971 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 0936d54d511a4cc3903264974a56ecb7 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-0936d54d511a4cc3903264974a56ecb7
I20260709 12:16:45.569388 18310 tablet_copy_source_session.cc:215] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:45.574402 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0936d54d511a4cc3903264974a56ecb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.606702 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:45.636404 19528 ts_tablet_manager.cc:933] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:45.647064 19520 tablet_copy_client.cc:806] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:45.647921 19520 tablet_copy_client.cc:670] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:45.651616 19528 tablet_copy_client.cc:323] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:45.653237 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet adea31c273544d04b7f16f9b9a3ae610 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:45.653731 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet adea31c273544d04b7f16f9b9a3ae610 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-adea31c273544d04b7f16f9b9a3ae610
I20260709 12:16:45.655336 19520 tablet_copy_client.cc:538] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:45.664320 19522 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:45.668135 19522 tablet_bootstrap.cc:492] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:45.677816 19037 raft_consensus.cc:1217] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:45.682101 19522 ts_tablet_manager.cc:1403] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.128s	user 0.039s	sys 0.029s
I20260709 12:16:45.685185 18385 tablet_copy_source_session.cc:215] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:45.693285 19528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adea31c273544d04b7f16f9b9a3ae610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.696774 19522 raft_consensus.cc:359] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.697633 19522 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:45.701864 19520 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:45.698305 19522 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.716873 19522 ts_tablet_manager.cc:1434] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:16:45.733253 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-4b135a527cce4b14b789d23618d411b0 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:45.733794 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-4b135a527cce4b14b789d23618d411b0 on tablet 4b135a527cce4b14b789d23618d411b0 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.759341 19528 tablet_copy_client.cc:806] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:45.760255 19528 tablet_copy_client.cc:670] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:45.764391 19528 tablet_copy_client.cc:538] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:45.766281 19520 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:45.774366 19520 tablet_bootstrap.cc:492] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:45.775146 19520 ts_tablet_manager.cc:1403] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.074s	user 0.023s	sys 0.001s
I20260709 12:16:45.778189 19520 raft_consensus.cc:359] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.791388 19520 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:45.792168 19520 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.794392 19528 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:45.795094 19520 ts_tablet_manager.cc:1434] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 12:16:45.796849 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0936d54d511a4cc3903264974a56ecb7 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:45.797240 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-0936d54d511a4cc3903264974a56ecb7 on tablet 0936d54d511a4cc3903264974a56ecb7 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.837118 19037 raft_consensus.cc:1217] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:45.869110 18447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:45.869784 18447 raft_consensus.cc:606] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.891872 19528 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:45.912735 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:45.914403 18369 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:45.916368 19528 tablet_bootstrap.cc:492] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:45.926928 19528 ts_tablet_manager.cc:1403] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.142s	user 0.025s	sys 0.000s
I20260709 12:16:45.936354 19528 raft_consensus.cc:359] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.938247 19528 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:45.939575 19528 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:45.955009 19528 ts_tablet_manager.cc:1434] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:16:45.956748 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-adea31c273544d04b7f16f9b9a3ae610 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:45.957211 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-adea31c273544d04b7f16f9b9a3ae610 on tablet adea31c273544d04b7f16f9b9a3ae610 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.967157 18294 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:45.967733 18294 raft_consensus.cc:606] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:45.970834 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:45.971475 18369 raft_consensus.cc:606] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:45.974450 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:45.974987 18369 raft_consensus.cc:606] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:45.978780 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:45.979835 18369 raft_consensus.cc:606] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:45.981132 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:16:45.982638 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:45.984193 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:46.040390 19499 raft_consensus.cc:993] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:46.040994 19499 raft_consensus.cc:1081] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:46.043049 19039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:46.043685 19039 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.044030 19039 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.054718 19039 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.057823 19464 raft_consensus.cc:993] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.058462 19531 raft_consensus.cc:1081] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.061522 18447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:46.062090 18447 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.062446 18447 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.062529 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:46.063248 18294 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.065335 19039 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:46.169173 18447 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.171909 18447 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:46.172204 18294 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:46.192386 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:46.204975 18291 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.194682 19039 raft_consensus.cc:1217] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:46.200811 19037 raft_consensus.cc:1217] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:46.218475 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:46.225793 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:46.228631 18369 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.228999 18369 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:46.229821 18369 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.233698 18369 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:46.299435 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 62953.4us
I20260709 12:16:46.301433 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:46.306043 19545 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:46.308525 19545 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:46.310019 19545 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.317900 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:46.318454 18446 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.318754 18446 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:46.319489 18446 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.324318 18446 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.325697 18369 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:46.330938 18446 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:46.331379 19502 raft_consensus.cc:1064] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:46.335016 19502 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.338918 19543 raft_consensus.cc:993] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.339561 19538 raft_consensus.cc:1081] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.345129 18197 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.344426 18446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:16:46.346724 18446 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.347141 18446 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.356160 18291 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:46.358330 19037 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:46.358810 18440 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:46.360625 19543 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:46.361513 19538 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:46.362309 18369 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:46.366426 19543 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:46.355052 18446 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.378290 18446 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:46.388089 19543 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:46.390098 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:46.391145 19542 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:46.390245 18445 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:46.391696 19542 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:46.392668 19542 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.403252 19531 raft_consensus.cc:993] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.403861 19531 raft_consensus.cc:1081] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.424382 19039 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:46.571377 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141286us
I20260709 12:16:46.634948 18194 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.676307 18198 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:46.700650 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 107515us
I20260709 12:16:46.744978 19547 raft_consensus.cc:993] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.745564 19531 raft_consensus.cc:1064] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:46.745692 19547 raft_consensus.cc:1081] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:46.748303 19531 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:16:46.753842 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 181620us
I20260709 12:16:46.755172 19464 raft_consensus.cc:993] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:46.755599 19464 raft_consensus.cc:1081] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
W20260709 12:16:46.783752 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104997us
I20260709 12:16:46.784994 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:46.785629 18370 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:46.791308 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62284.9us
I20260709 12:16:46.846045 18444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:46.846679 18444 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.847059 18444 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.848453 18449 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:46.848955 18449 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.849375 18449 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.831034 18370 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:46.870923 18444 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.871747 18449 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.875551 18447 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:46.874317 19038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59430
I20260709 12:16:46.878976 19038 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:46.902580 19038 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.903402 18441 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:46.913208 19038 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.919117 19040 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:46.918608 18298 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.915546 18449 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:46.914927 18444 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:46.938939 19495 raft_consensus.cc:1064] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
W20260709 12:16:46.940649 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54887.1us
I20260709 12:16:46.950109 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:46.950806 18366 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.951139 18366 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:46.948035 18373 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:46.951974 18366 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.953418 18447 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:46.953961 18366 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.954846 19545 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:46.956967 18444 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:46.949425 19495 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:46.961406 19038 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:46.963303 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:46.969209 19039 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:46.970171 18372 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:46.971345 19516 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:46.971841 19516 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:46.971946 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:46.972517 18374 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.972854 18374 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:46.972715 19516 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.977859 19040 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:46.981164 18294 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:46.973313 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:46.981688 19452 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:46.981730 18372 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.982867 18372 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:46.985855 18449 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:46.984761 19564 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:16:46.985982 19544 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:46.986418 19544 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:46.987149 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:46.987752 18446 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.989475 19452 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:46.987856 19544 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.993654 19544 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.994295 18374 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:46.995918 18372 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:47.001037 19567 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:47.004534 18372 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.003945 19497 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:16:47.009852 18444 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:47.014458 18443 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:47.043606 19561 raft_consensus.cc:1064] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:47.051215 19497 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:47.069238 19573 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:47.070783 19566 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:16:47.073696 19546 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 12:16:47.085073 19571 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:16:47.111416 18374 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:47.116430 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 94447.9us
I20260709 12:16:47.126672 18375 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:47.131982 18300 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:47.020085 19040 raft_consensus.cc:2955] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:47.148130 18446 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:47.151774 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:47.152801 19545 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:47.193478 18366 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:47.194684 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:47.197034 18374 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:47.198017 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:47.195271 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9f2e3c205c14e43b532c52bc7fa4c90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:47.199489 19566 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:16:47.200573 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:47.199503 18372 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:47.200268 19540 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:47.223896 18197 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.205688 19566 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:47.239779 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:47.267997 18297 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.268639 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:47.269181 18295 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.273466 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:47.273973 18297 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.276016 18295 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:47.283519 18297 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:47.286566 18443 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:47.335335 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 64560.1us
I20260709 12:16:47.303539 19568 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:16:47.296901 19561 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:47.353324 18194 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.301666 19566 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:47.381839 19585 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 12:16:47.382997 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 122410us
I20260709 12:16:47.383255 19540 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:47.384375 19540 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:47.345976 19545 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:16:47.392843 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61701.2us
W20260709 12:16:47.398186 19575 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:16:47.398407 19545 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:47.355458 19582 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:16:47.407963 19039 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:47.468510 19551 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:16:47.472983 19577 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:16:47.491381 18300 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
W20260709 12:16:47.494992 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 79623.9us
I20260709 12:16:47.411895 19498 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:47.606052 19569 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
W20260709 12:16:47.650013 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 203207us
I20260709 12:16:47.717949 18297 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:47.727089 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 268333us
W20260709 12:16:47.746114 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 317574us
I20260709 12:16:47.768847 18300 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:47.772365 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 341980us
I20260709 12:16:47.833074 19042 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:47.843173 18198 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.853121 18373 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.856257 18195 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:47.850687 18369 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:16:47.907644 19595 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:16:47.927314 18372 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:47.917995 19545 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:16:47.944162 19581 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:16:47.946301 19545 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:47.986413 19545 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:47.990006 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:47.995474 18448 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:48.008783 19581 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:16:48.026218 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 165557us
I20260709 12:16:48.039383 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:48.042465 19452 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:48.049953 19039 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.053925 19557 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:16:48.058024 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 431303us
I20260709 12:16:48.066573 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet b9f2e3c205c14e43b532c52bc7fa4c90 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:48.069558 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:48.104619 19612 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:48.122645 18295 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:48.143440 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 373319us
I20260709 12:16:48.187064 19597 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:48.206442 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 168883us
I20260709 12:16:48.227128 19564 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:48.234773 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:48.236259 19581 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:48.222504 18195 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:48.238446 18195 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:48.249852 19625 tablet_replica.cc:333] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:48.281689 19625 raft_consensus.cc:2243] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:48.266789 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 50636.8us
I20260709 12:16:48.305213 18367 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:48.313000 19625 raft_consensus.cc:2272] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.304883 18374 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:48.308190 18371 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:48.331321 18297 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:48.355491 18444 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.349393 19562 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:16:48.362047 19625 ts_tablet_manager.cc:1916] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:48.349777 19597 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:48.338222 18448 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:48.458277 19452 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:48.458855 19452 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:48.468508 19452 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:48.471961 19040 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:48.473340 18374 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:48.423714 19597 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:48.484690 19597 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:48.485591 19542 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 12:16:48.481014 19571 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:16:48.503561 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:48.515844 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:48.521831 19542 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:48.568027 19561 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:16:48.573724 19516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.169s	sys 0.236s
W20260709 12:16:48.574265 19516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.169s	sys 0.237s
I20260709 12:16:48.579396 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:48.596938 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:48.616371 19540 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:16:48.701431 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:48.643855 18443 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:48.827270 19044 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:48.828933 19598 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:48.850179 19547 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:16:48.850855 19547 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:48.855710 19547 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:48.922619 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:48.943202 19598 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:48.990336 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 48215.7us
W20260709 12:16:49.014173 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 398943us
I20260709 12:16:49.018328 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:49.012512 19542 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:49.019372 19542 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:49.021311 19542 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:49.021477 19538 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.008543 19590 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:49.022938 19590 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:49.025030 19590 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:49.007668 19581 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:16:49.030431 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.026119 19581 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.042363 19041 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.047086 19581 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:49.048458 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.035895 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.050432 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:16:49.050678 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:49.047938 19573 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.054628 18294 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.057511 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.060575 19612 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:49.061053 19612 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:49.061017 18197 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.068856 19612 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:49.070317 19581 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.075817 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b135a527cce4b14b789d23618d411b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:16:49.087107 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54076.1us
I20260709 12:16:49.090461 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.094885 19625 ts_tablet_manager.cc:1929] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:49.104102 19625 log.cc:1199] T b9f2e3c205c14e43b532c52bc7fa4c90 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/b9f2e3c205c14e43b532c52bc7fa4c90
I20260709 12:16:49.104823 19635 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:16:49.135442 18447 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.156997 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:16:49.169737 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104192us
I20260709 12:16:49.153833 19569 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.175508 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:49.147964 19635 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.191823 18447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37ef0235d410491ab47198988075f6e4"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:49.192469 18447 raft_consensus.cc:606] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.184386 19569 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:49.220441 18339 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.222512 19598 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:49.226946 19546 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.189852 19625 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46412 (request call id 75) took 1124 ms. Trace:
I20260709 12:16:49.261711 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:49.256868 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:49.294950 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:16:49.307431 19569 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:49.309069 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:16:49.320185 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.319826 18443 raft_consensus.cc:2955] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.310180 19635 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:49.329826 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.331290 19542 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.335722 18412 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.341357 19516 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.342103 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.342818 19582 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.253854 19625 rpcz_store.cc:276] 0709 12:16:48.058113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:48.066407 (+  8294us) service_pool.cc:224] Handling call
0709 12:16:48.249268 (+182861us) ts_tablet_manager.cc:1151] Deleting tablet b9f2e3c205c14e43b532c52bc7fa4c90
0709 12:16:48.249350 (+    82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:16:48.771799 (+522449us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:49.094710 (+322911us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:49.177436 (+ 82726us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":173610,"thread_start_us":173208,"threads_started":1}
I20260709 12:16:49.343204 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:49.347398 19635 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.352360 19042 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.364277 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet b9f2e3c205c14e43b532c52bc7fa4c90 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:49.369740 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:49.374794 19569 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:16:49.428903 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:16:49.462880 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:16:49.522032 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52653.8us
I20260709 12:16:49.527405 18447 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.527015 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.529527 19540 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:49.538080 18448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:49.538626 18448 raft_consensus.cc:606] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.534072 19638 raft_consensus.cc:993] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.555526 19638 raft_consensus.cc:1081] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.561141 19042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37ef0235d410491ab47198988075f6e4"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:49.561755 19042 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:49.562135 19042 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.575246 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:49.575903 18366 raft_consensus.cc:606] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.577207 19042 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.584636 19649 raft_consensus.cc:993] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.585209 19649 raft_consensus.cc:1081] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.589887 19042 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:49.591514 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:49.592082 18371 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.595810 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:49.597007 18443 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:49.597394 18443 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:49.598269 18443 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.600284 18443 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.610349 18296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:49.618518 18296 raft_consensus.cc:606] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.621393 18449 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:49.621923 18449 raft_consensus.cc:606] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.624516 18449 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:49.625023 18449 raft_consensus.cc:606] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:49.626262 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:16:49.617784 18371 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.627970 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:49.628079 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:49.628976 19631 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.629447 19631 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:49.628140 19038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:49.630213 19038 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:49.630635 19038 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.630313 19631 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.634413 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:49.640640 19612 raft_consensus.cc:993] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.641117 19612 raft_consensus.cc:1081] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.643548 19040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:49.644125 19040 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:49.644454 19040 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.658746 19038 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.662627 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:49.662462 19038 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:49.670615 19040 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.667018 18447 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:49.673563 18447 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:49.673569 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:49.674233 18366 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.674438 18447 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.675577 18447 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.665953 18443 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.703053 18197 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 131) took 1173 ms. Trace:
I20260709 12:16:49.789496 19564 raft_consensus.cc:993] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.790040 19564 raft_consensus.cc:1081] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:49.787809 18197 rpcz_store.cc:276] 0709 12:16:48.529403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:48.529781 (+   378us) service_pool.cc:224] Handling call
0709 12:16:49.039594 (+509813us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b5400002490. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d580b2 00007f3c6ecd1759 00007f3c6ec69a76 00007f3c6ec3ad58 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc
0709 12:16:49.702968 (+663374us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:49.187218 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:49.187889 (+   671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:49.187940 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:16:49.187970 (+    30us) tablet.cc:662] Decoding operations
0709 12:16:49.209041 (+ 21071us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:49.216747 (+  7706us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:49.244474 (+ 27727us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:49.255541 (+ 11067us) tablet.cc:701] Row locks acquired
0709 12:16:49.255562 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:16:49.255698 (+   136us) write_op.cc:270] Start()
0709 12:16:49.255804 (+   106us) write_op.cc:276] Timestamp: P: 1783599409255660 usec, L: 0
0709 12:16:49.255829 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:16:49.256475 (+   646us) log.cc:844] Serialized 599 byte log entry
0709 12:16:49.640310 (+383835us) op_driver.cc:464] REPLICATION: finished
0709 12:16:49.700267 (+ 59957us) write_op.cc:301] APPLY: starting
0709 12:16:49.700345 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:49.700431 (+    86us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:49.700529 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:49.700731 (+   202us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:49.700867 (+   136us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:16:49.700907 (+    40us) write_op.cc:312] APPLY: finished
0709 12:16:49.701075 (+   168us) log.cc:844] Serialized 23 byte log entry
0709 12:16:49.701235 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:49.701301 (+    66us) write_op.cc:454] Released schema lock
0709 12:16:49.701339 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:49.701591 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:49.701613 (+    22us) write_op.cc:454] Released schema lock
0709 12:16:49.762938 (+ 61325us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7932a 00007f3c68dfef14 00007f3c68dfeec3 00007f3c68dfee38 00007f3c68dfedf3 00007f3c68dfe0dc 00007f3c68e27e16 00007f3c68e0fcbb 00007f3c68dfa39d 00007f3c68dfdd75
Metrics: {"mutex_wait_us":1026,"reported_tablets":1,"spinlock_wait_cycles":96771968,"child_traces":[["op",{"apply.queue_time_us":59326,"apply.run_cpu_time_us":1371,"apply.run_wall_time_us":1465,"mutex_wait_us":6590,"num_ops":1,"prepare.queue_time_us":117856,"prepare.run_cpu_time_us":8445,"prepare.run_wall_time_us":149267,"raft.queue_time_us":180699,"raft.run_cpu_time_us":4958,"raft.run_wall_time_us":176258,"replication_time_us":384080,"spinlock_wait_cycles":121903488,"thread_start_us":360218,"threads_started":4,"wal-append.queue_time_us":119777}]]}
I20260709 12:16:49.696450 18447 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.798530 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:49.799294 19631 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.799767 19631 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:49.800565 19631 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.690805 19040 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:49.710407 18194 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.684121 18366 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.803030 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:49.802870 18194 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.804476 19581 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.804927 19581 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:49.682631 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:49.714228 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:49.806141 18294 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.805778 19581 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.733546 18293 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.791754 19038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59426
I20260709 12:16:49.810178 19038 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:49.810621 19038 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.813308 18294 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.814483 18449 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:49.814886 18449 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:49.815833 18449 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.817348 18449 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.824473 18449 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.839174 18366 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.841426 18198 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.850793 18371 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.851027 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b135a527cce4b14b789d23618d411b0 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:49.857471 19589 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:49.859163 19040 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.861171 19607 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:49.869700 19038 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.872785 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:49.873320 18449 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.875628 19652 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.882122 19042 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:49.883303 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:49.883744 18449 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.883852 18294 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:49.884227 18294 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:49.884912 18294 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.885880 18294 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.891979 19039 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.893177 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:49.894184 19581 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.894722 19581 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:49.896698 19581 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:49.899746 18294 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:49.903889 19038 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:49.924361 19671 tablet_replica.cc:333] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:16:49.921449 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118488us
I20260709 12:16:49.936496 18294 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.914017 19495 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:16:49.929608 19671 raft_consensus.cc:2243] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.939074 19671 raft_consensus.cc:2272] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.940949 19559 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 12:16:50.030047 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 76216.4us
W20260709 12:16:50.044548 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 88329.9us
W20260709 12:16:50.049598 19670 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.094s	user 0.002s	sys 0.007s
I20260709 12:16:50.067322 18198 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:50.074530 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 119839us
I20260709 12:16:50.075757 18198 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.082233 19671 ts_tablet_manager.cc:1916] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:50.084233 19589 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:50.091399 18449 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.092073 18372 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.100637 19668 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:16:50.102980 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54600.7us
I20260709 12:16:50.105149 19631 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 12:16:50.179913 19672 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:16:50.177412 18198 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.182595 18198 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:50.150830 19042 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:50.285483 18294 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.321211 19581 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:50.321913 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0936d54d511a4cc3903264974a56ecb7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:50.347934 18195 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:50.359577 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet adea31c273544d04b7f16f9b9a3ae610 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:50.361892 18371 raft_consensus.cc:2955] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:50.367463 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 261038us
I20260709 12:16:50.402410 18371 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.403992 19631 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:50.422147 18443 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.425474 19631 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:50.429409 18443 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.438253 19671 ts_tablet_manager.cc:1929] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:50.438835 19671 log.cc:1199] T 4b135a527cce4b14b789d23618d411b0 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/4b135a527cce4b14b789d23618d411b0
I20260709 12:16:50.463845 19631 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:50.469319 19607 raft_consensus.cc:993] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
W20260709 12:16:50.472882 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 185557us
I20260709 12:16:50.502099 19573 raft_consensus.cc:1081] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:50.473462 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet adea31c273544d04b7f16f9b9a3ae610 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:50.474855 18436 tablet_service.cc:1558] Processing DeleteTablet for tablet 0936d54d511a4cc3903264974a56ecb7 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:50.504624 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 4b135a527cce4b14b789d23618d411b0 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
W20260709 12:16:50.476758 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 216073us
I20260709 12:16:50.478791 19612 raft_consensus.cc:993] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:50.510854 19612 raft_consensus.cc:1081] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:50.527864 19044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59430
I20260709 12:16:50.528123 18298 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.528486 19044 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.528909 19044 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.530629 19631 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:50.531965 18293 raft_consensus.cc:2955] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:50.538012 19044 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:50.543354 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:50.543867 18367 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.544133 18367 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:50.544759 18367 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:50.545545 18367 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.561838 19044 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:50.564512 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:50.568919 18447 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:50.582698 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:50.583647 19668 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.584105 19668 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:50.584965 19668 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:50.585835 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:50.586387 18441 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.593324 19688 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:50.573928 19044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:50.599390 19044 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.599877 19044 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.616144 18441 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:50.629958 19695 tablet_replica.cc:333] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:50.631124 19695 raft_consensus.cc:2243] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:50.644482 19695 raft_consensus.cc:2272] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:50.658224 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:50.673319 19044 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:16:50.691088 19691 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.064s	user 0.004s	sys 0.000s
I20260709 12:16:50.695948 19695 ts_tablet_manager.cc:1916] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:50.713909 19040 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:50.719647 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:50.720186 18293 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.720677 19678 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:50.721705 19680 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.722074 19680 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:50.730139 18198 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.731633 19680 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:50.732636 19680 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.748152 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:50.750386 19044 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:50.781857 18441 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:50.791034 18293 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:50.791088 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:50.793633 19668 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.797658 19668 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:50.798563 19668 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:50.822878 19695 ts_tablet_manager.cc:1929] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:50.823403 19695 log.cc:1199] T adea31c273544d04b7f16f9b9a3ae610 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/adea31c273544d04b7f16f9b9a3ae610
I20260709 12:16:50.825033 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet adea31c273544d04b7f16f9b9a3ae610 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:50.826850 19695 tablet_replica.cc:333] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:50.827978 19695 raft_consensus.cc:2243] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:50.828738 19695 raft_consensus.cc:2272] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:50.828225 18198 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.831395 19695 ts_tablet_manager.cc:1916] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:50.851415 19695 ts_tablet_manager.cc:1929] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:50.851830 19695 log.cc:1199] T 0936d54d511a4cc3903264974a56ecb7 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/0936d54d511a4cc3903264974a56ecb7
I20260709 12:16:50.853257 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 0936d54d511a4cc3903264974a56ecb7 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:51.072434 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:51.093765 18446 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:51.095283 19688 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.099664 18371 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:51.102198 19688 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.211165 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:51.211894 18371 raft_consensus.cc:606] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:51.217159 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:51.217765 18371 raft_consensus.cc:606] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:51.220577 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:51.221282 18371 raft_consensus.cc:606] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:51.229737 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:51.230406 18371 raft_consensus.cc:606] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:51.232400 18441 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:51.234282 19688 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.248131 18298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:51.249279 18298 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:51.253360 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:51.255399 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:51.256274 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:51.275439 18298 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:51.279356 19688 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.299861 19567 raft_consensus.cc:993] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:51.300323 19567 raft_consensus.cc:1081] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:51.302027 18371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:16:51.302527 18371 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.302846 18371 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.313467 18371 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.318992 18371 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:51.322324 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:16:51.322937 19044 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.327188 19044 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:16:51.329706 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:16:51.330467 19652 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:51.331151 19652 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:51.332280 19652 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.338913 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:51.339537 18294 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:51.339828 18294 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:51.340481 18294 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.341702 18294 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:51.351188 18294 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:16:51.414920 18195 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.449292 19686 raft_consensus.cc:993] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:51.449820 19686 raft_consensus.cc:1081] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:51.459615 18298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:45180
I20260709 12:16:51.460150 18298 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.460456 18298 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.478021 19559 raft_consensus.cc:993] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:51.478511 19559 raft_consensus.cc:1081] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:51.479017 18298 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.481698 18298 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:51.481762 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:51.482885 18367 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.483178 18367 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:51.483923 18367 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.484899 18367 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.484849 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:51.489290 18441 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.496126 18367 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:51.497334 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:51.498309 19567 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.503733 19040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59430
I20260709 12:16:51.504413 19040 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.504838 19040 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.507345 18441 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:51.511741 19567 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:51.512676 19567 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.522050 19040 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.530857 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:51.531358 18294 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.538638 18294 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:51.541311 19040 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:51.542512 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:51.542714 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:51.543076 18367 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.543459 18367 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:51.544138 18367 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.544621 19688 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.545166 19688 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:51.545326 18367 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.546132 19688 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:51.551332 18367 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:51.559065 18195 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.565383 18294 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:51.568924 18194 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.592510 19559 raft_consensus.cc:993] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:51.593071 19652 raft_consensus.cc:1081] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:51.596131 18448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:51.596864 18448 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.597380 18448 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.607754 18448 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.610639 18448 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:51.610791 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:51.611207 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:51.611966 18298 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.611970 18367 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.614530 18367 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:51.615183 18367 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.617542 18367 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.622555 18441 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:51.623071 18263 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e2ee63a449648bfb064fadee177cadb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.625262 18367 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:51.627738 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:51.627857 19564 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.628885 19678 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.629429 19559 raft_consensus.cc:993] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:51.629379 19678 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:51.629997 19559 raft_consensus.cc:1081] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:51.630517 19678 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.633479 18448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:51.634135 18448 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:51.634610 18448 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.637370 18367 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.637811 18298 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:51.639638 19710 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.643270 18448 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.647980 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:51.648273 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:51.648952 18371 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:51.649259 18371 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:51.650048 18371 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.650988 18371 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.651897 19564 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.653776 18441 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.654208 18194 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.657441 18371 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:51.659829 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:51.660648 19678 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:51.661093 19678 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:51.648538 18298 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:51.662926 19678 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:51.672493 18298 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:51.671419 18366 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.676035 18448 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:51.675038 18367 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:51.686455 18195 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.765094 18298 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.771749 19652 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:51.772984 18338 consensus_peers.cc:597] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6021c985b4e44fe880350a321132a9b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.773450 18441 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.775502 19682 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.784654 18194 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.815120 19725 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.817512 18441 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.817533 18294 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.838373 18194 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.839440 18366 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:16:51.871874 18338 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 353cbd244a354830a4e903a3b1be2b89. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.872900 18441 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.873596 18298 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.874684 19682 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.877468 19682 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.910377 18448 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.912978 19564 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.916473 19724 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.934348 18446 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.953274 18198 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.975373 18294 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:51.985154 18371 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.985154 18367 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:51.987874 19044 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:16:51.988902 18293 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:16:51.988790 18298 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:51.989415 19686 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.990496 19727 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:51.992630 19710 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.993019 19688 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:52.013310 18448 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:52.014554 19710 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:52.018680 18371 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:52.019980 18263 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8bf6d66a261490caa69a80b24ea015f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.029521 19567 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.042336 18300 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:52.043795 19666 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:52.066341 19567 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:52.078789 18372 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:52.104363 18371 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:52.105801 19678 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.146303 18448 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:52.130959 18298 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:52.153663 19678 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.195178 18197 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:52.235311 19749 ts_tablet_manager.cc:933] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:52.254374 19749 tablet_copy_client.cc:323] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:52.254668 18366 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:52.256700 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 6021c985b4e44fe880350a321132a9b5 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:52.257469 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 6021c985b4e44fe880350a321132a9b5 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-6021c985b4e44fe880350a321132a9b5
I20260709 12:16:52.262178 19678 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:52.262897 19750 ts_tablet_manager.cc:933] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:52.277566 19750 tablet_copy_client.cc:323] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:52.279181 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 9e2ee63a449648bfb064fadee177cadb from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:52.279724 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 9e2ee63a449648bfb064fadee177cadb from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-9e2ee63a449648bfb064fadee177cadb
I20260709 12:16:52.290424 18300 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:52.293340 18310 tablet_copy_source_session.cc:215] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.293891 19689 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:52.301846 19750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2ee63a449648bfb064fadee177cadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.310782 18385 tablet_copy_source_session.cc:215] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.333076 19749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6021c985b4e44fe880350a321132a9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.367872 19749 tablet_copy_client.cc:806] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.374107 19749 tablet_copy_client.cc:670] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.378159 19749 tablet_copy_client.cc:538] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.386972 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:52.402922 19750 tablet_copy_client.cc:806] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.406060 19750 tablet_copy_client.cc:670] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.409909 19750 tablet_copy_client.cc:538] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.404493 19749 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:52.434382 19750 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:52.436331 19755 ts_tablet_manager.cc:933] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:52.452627 19755 tablet_copy_client.cc:323] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:16:52.458052 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 353cbd244a354830a4e903a3b1be2b89 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:16:52.458630 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 353cbd244a354830a4e903a3b1be2b89 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-353cbd244a354830a4e903a3b1be2b89
I20260709 12:16:52.464442 19749 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.465473 19749 tablet_bootstrap.cc:492] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:52.466223 19749 ts_tablet_manager.cc:1403] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.001s
I20260709 12:16:52.470145 19749 raft_consensus.cc:359] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.478051 19749 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:52.478775 19749 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.481194 19749 ts_tablet_manager.cc:1434] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.003s
I20260709 12:16:52.477720 19750 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.471503 18385 tablet_copy_source_session.cc:215] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.483748 19750 tablet_bootstrap.cc:492] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:52.484498 19750 ts_tablet_manager.cc:1403] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.011s
I20260709 12:16:52.485987 19755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353cbd244a354830a4e903a3b1be2b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.488054 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-6021c985b4e44fe880350a321132a9b5 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:52.488485 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-6021c985b4e44fe880350a321132a9b5 on tablet 6021c985b4e44fe880350a321132a9b5 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:52.492096 19750 raft_consensus.cc:359] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.493078 19750 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:52.495445 19750 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.498196 19750 ts_tablet_manager.cc:1434] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:52.499809 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-9e2ee63a449648bfb064fadee177cadb received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:52.500216 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-9e2ee63a449648bfb064fadee177cadb on tablet 9e2ee63a449648bfb064fadee177cadb with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:52.506925 19755 tablet_copy_client.cc:806] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.507548 19755 tablet_copy_client.cc:670] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.512271 19755 tablet_copy_client.cc:538] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.526260 19749 ts_tablet_manager.cc:933] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:52.531685 19755 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:52.545921 19749 tablet_copy_client.cc:323] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:52.547729 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet d8bf6d66a261490caa69a80b24ea015f from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:52.548254 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet d8bf6d66a261490caa69a80b24ea015f from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-d8bf6d66a261490caa69a80b24ea015f
I20260709 12:16:52.556046 18310 tablet_copy_source_session.cc:215] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.570497 19749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bf6d66a261490caa69a80b24ea015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.570752 19755 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.576683 19755 tablet_bootstrap.cc:492] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:52.583058 19755 ts_tablet_manager.cc:1403] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.003s
I20260709 12:16:52.588904 19755 raft_consensus.cc:359] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.589778 19755 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:52.590449 19755 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.590905 18448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:52.595552 18448 raft_consensus.cc:606] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:52.599529 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:52.600099 18293 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:52.607290 19755 ts_tablet_manager.cc:1434] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:16:52.612438 18448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:52.613003 18448 raft_consensus.cc:606] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:52.615025 19040 raft_consensus.cc:1217] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.617154 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-353cbd244a354830a4e903a3b1be2b89 received from {username='slave'} at 127.0.0.1:56140
I20260709 12:16:52.617631 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-353cbd244a354830a4e903a3b1be2b89 on tablet 353cbd244a354830a4e903a3b1be2b89 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:52.619951 18367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:52.620483 18367 raft_consensus.cc:606] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:52.621915 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:52.623595 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:52.624359 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:52.629588 19749 tablet_copy_client.cc:806] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.631096 19749 tablet_copy_client.cc:670] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.634923 19749 tablet_copy_client.cc:538] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.644256 19749 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:52.693974 19678 raft_consensus.cc:993] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:52.694543 19678 raft_consensus.cc:1081] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:52.698673 18366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:46418
I20260709 12:16:52.699326 18366 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:52.699918 18366 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:52.729035 18366 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:52.742097 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:52.742704 18448 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:52.742980 18448 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:52.743705 18448 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:52.744791 18448 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:52.747990 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:52.748648 18293 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:52.763190 18448 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:52.768633 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:52.769404 19724 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:52.769892 19724 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:52.766866 18293 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:52.767333 18366 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:52.770690 19724 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:52.774761 19040 raft_consensus.cc:1217] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.792927 18196 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:52.808352 19749 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.812669 19749 tablet_bootstrap.cc:492] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:52.813462 19749 ts_tablet_manager.cc:1403] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.169s	user 0.024s	sys 0.005s
I20260709 12:16:52.816586 19749 raft_consensus.cc:359] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.829000 19749 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:52.829924 19749 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:52.836669 19749 ts_tablet_manager.cc:1434] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 12:16:52.838306 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-d8bf6d66a261490caa69a80b24ea015f received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:52.838721 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-d8bf6d66a261490caa69a80b24ea015f on tablet d8bf6d66a261490caa69a80b24ea015f with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:52.849907 19685 raft_consensus.cc:993] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:52.850404 19685 raft_consensus.cc:1081] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:52.855535 19040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59422
I20260709 12:16:52.856388 19040 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:52.856927 19040 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:52.884732 19040 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:52.886868 19040 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:52.887926 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:52.888509 18448 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:52.888769 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:52.889397 18300 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:52.888792 18448 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:52.896327 18448 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:52.897589 18448 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:52.944787 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:52.948652 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:52.949646 19746 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:52.950173 19746 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:52.951289 19746 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:52.958914 18448 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:16:52.984942 19044 raft_consensus.cc:1217] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.966854 18196 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.016580 19724 raft_consensus.cc:993] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:53.017447 19652 raft_consensus.cc:1081] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:53.019410 18446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:53.020051 18446 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:53.020459 18446 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.068691 19727 raft_consensus.cc:1064] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:53.071811 19727 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:53.139046 19040 raft_consensus.cc:1217] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:53.158459 18300 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.160060 19652 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.169716 19040 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.171478 19724 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.179199 18448 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.181603 19725 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.182310 18300 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:53.183830 19770 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:53.186338 18446 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:53.211011 18446 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:53.222078 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:53.225839 18298 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.226770 19763 raft_consensus.cc:993] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:53.227253 19763 raft_consensus.cc:1081] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:53.213011 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:53.232170 18367 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:53.232478 18367 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:53.237710 18298 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:53.259879 18367 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:53.262161 18367 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.284263 18366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:16:53.285092 18366 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:53.285526 18366 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.298755 19763 raft_consensus.cc:1064] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
W20260709 12:16:53.299185 19763 raft_consensus.cc:1068] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Unable to promote non-voter 991dcd2a70c74fdb89ca9494cc09fb7b: Service unavailable: leader transfer in progress
I20260709 12:16:53.302058 18366 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:53.302665 19040 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:53.306296 18366 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:53.306506 18367 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:53.308804 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:53.308878 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:53.314158 19767 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:53.323933 19767 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:53.324844 19767 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:16:53.328562 18293 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:53.328966 18293 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:53.329738 18293 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:53.331221 18293 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.334601 18448 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:53.305002 19724 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:53.338825 18196 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:53.340525 19774 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:16:53.346228 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 130078us
I20260709 12:16:53.349856 18447 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:53.355093 18293 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:16:53.356801 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:53.362044 18447 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.372551 18338 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:53.365664 19727 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:16:53.387295 18447 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:16:53.379107 19652 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:53.395660 19652 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:53.396559 19652 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:53.413089 18293 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:53.421809 19725 raft_consensus.cc:1064] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:53.426342 19725 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:53.418313 19774 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 12:16:53.472025 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 110273us
W20260709 12:16:53.474967 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 109509us
I20260709 12:16:53.531006 18198 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:53.558432 19789 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:16:53.570443 19784 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
W20260709 12:16:53.603415 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98353.8us
I20260709 12:16:53.645723 18293 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:53.691941 18446 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:53.704311 19038 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.727631 19783 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.727662 18444 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.760821 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 2 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.775852 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:53.783068 19792 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:16:53.794059 18300 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:53.854274 19724 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.863505 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:53.877895 18366 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Received request to transfer leadership
I20260709 12:16:53.883126 18296 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:53.884821 19727 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.911900 18298 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:53.923914 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:53.924700 18366 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Received request to transfer leadership
W20260709 12:16:53.926900 18185 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 9e2ee63a449648bfb064fadee177cadb already in progress [suppressed 2 similar messages]
I20260709 12:16:53.940354 18441 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:53.950942 18366 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:53.971195 19727 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:16:53.983974 18373 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:53.994756 19791 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:54.006714 18185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78467.5us
I20260709 12:16:54.004907 19791 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:16:54.018309 19795 raft_consensus.cc:993] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:54.018791 19795 raft_consensus.cc:1081] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:54.033687 18447 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.038610 18298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:45180
I20260709 12:16:53.985495 19776 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.039276 18298 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.039969 18298 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.040589 18294 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.044327 19040 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Advancing to term 2
I20260709 12:16:54.044476 19806 raft_consensus.cc:1064] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:54.049573 18298 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:54.055367 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:54.060185 19040 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:54.068163 18296 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:16:54.079572 18298 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:54.092782 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:54.073983 19783 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:16:54.097834 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:54.098321 18441 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.105674 18441 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:16:54.108633 19811 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:16:54.109723 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:54.111500 19804 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:54.112197 19804 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:54.135988 18369 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:54.136377 18369 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:54.138494 18441 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:16:54.143126 18369 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:54.144340 18369 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:54.161430 18196 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.178751 19804 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:54.184729 18369 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:16:54.185395 18371 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 12:16:54.185838 18371 raft_consensus.cc:614] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:54.195972 18447 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:16:54.225445 18246 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:16:54.227298 18246 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:16:54.229063 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 122906us
I20260709 12:16:54.122938 19806 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.235059 19040 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:16:54.227614 18246 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:16:54.243207 19038 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.278239 19793 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:16:54.293326 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:54.293974 18366 raft_consensus.cc:606] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:54.295308 19724 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.310135 18298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d4d1763576e4c49849611360f37665a"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:54.310761 18298 raft_consensus.cc:606] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:54.313472 19044 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.324119 18366 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.325461 18447 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.328400 18446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:54.328910 18446 raft_consensus.cc:606] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:16:54.329279 18185 catalog_manager.cc:4755] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): ChangeConfig:REMOVE_PEER RPC failed for tablet 353cbd244a354830a4e903a3b1be2b89: Remote error: Service unavailable: leader transfer in progress
I20260709 12:16:54.336647 18198 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 2 to 3, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.327194 19763 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:16:54.341302 19808 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:54.345047 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:16:54.346961 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:54.352696 19804 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:16:54.363356 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:54.360536 19725 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.387779 19038 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:54.395931 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56197.5us
I20260709 12:16:54.440908 18446 raft_consensus.cc:2955] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.453671 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6021c985b4e44fe880350a321132a9b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.501098 18196 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 1 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.519385 19808 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.521365 19038 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.548409 18371 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.561054 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 6021c985b4e44fe880350a321132a9b5 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:54.566344 19040 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Advancing to term 3
I20260709 12:16:54.578207 19040 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:54.583431 19763 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:16:54.585229 18198 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.621336 19816 raft_consensus.cc:993] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:54.621810 19816 raft_consensus.cc:1081] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:54.624315 18441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:54.624846 18441 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.625160 18441 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.628059 19798 raft_consensus.cc:1064] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
W20260709 12:16:54.628516 19798 raft_consensus.cc:1068] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Unable to promote non-voter 991dcd2a70c74fdb89ca9494cc09fb7b: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:16:54.633052 18447 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.645102 18371 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:54.646843 19564 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:54.659344 19796 raft_consensus.cc:993] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:54.659878 19796 raft_consensus.cc:1081] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:54.661656 18298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:16:54.662148 18298 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.662494 18298 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.673452 18298 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:54.675056 18446 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:54.675689 19829 tablet_replica.cc:333] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:54.654860 18300 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.692658 19808 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:54.679991 19804 raft_consensus.cc:1064] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:54.696203 18298 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:54.696883 18446 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.696729 19804 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.637730 18441 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.701040 19763 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.702005 18441 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:54.702584 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:54.703246 18446 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:54.703568 18446 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:54.704559 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:54.705075 18366 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:54.705397 18366 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:54.705292 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:54.706333 18369 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.707895 19040 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.709564 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:54.710090 18298 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.712688 18366 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.711935 19564 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:16:54.716822 18366 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.718691 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:16:54.720697 19038 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.721411 19808 raft_consensus.cc:993] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:54.721910 19808 raft_consensus.cc:1081] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:54.721942 19829 raft_consensus.cc:2243] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.723635 19829 raft_consensus.cc:2272] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.725641 18446 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:54.730654 18298 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:54.732316 18371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d4d1763576e4c49849611360f37665a"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:16:54.732824 18371 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.733153 18371 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.734979 18446 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.743683 18446 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:54.748685 18367 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:16:54.749282 19798 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.752151 19044 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.745961 18371 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.757403 19829 ts_tablet_manager.cc:1916] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.768456 19040 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:16:54.770958 18366 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:54.767151 18369 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:54.779625 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:54.782380 18371 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:54.783586 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:54.790529 19821 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:54.791019 19821 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:54.791951 19821 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.776873 19804 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:16:54.793193 19564 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:54.790035 19808 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:54.795590 19808 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:54.796440 19808 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:54.820255 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:54.821013 18298 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:54.821379 18298 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:54.832252 18446 raft_consensus.cc:2955] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:54.841302 18185 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 353cbd244a354830a4e903a3b1be2b89 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.834758 18298 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.863701 18298 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.895624 18298 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
W20260709 12:16:54.900635 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54571.6us
W20260709 12:16:54.903452 19785 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
I20260709 12:16:54.905879 19829 ts_tablet_manager.cc:1929] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.907727 19829 log.cc:1199] T 6021c985b4e44fe880350a321132a9b5 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/6021c985b4e44fe880350a321132a9b5
I20260709 12:16:54.911538 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:54.911938 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:16:54.912636 19040 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.915321 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8bf6d66a261490caa69a80b24ea015f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.916791 19038 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:54.918298 19724 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:54.918831 19724 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:54.922955 19724 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:54.924674 19564 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.926767 18367 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.929024 19040 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:16:54.933754 18194 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.934893 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 6021c985b4e44fe880350a321132a9b5 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:54.944586 19040 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.957185 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet d8bf6d66a261490caa69a80b24ea015f with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:54.963964 18441 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:16:54.958462 18198 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.968741 19804 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:54.968837 18198 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 5, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.975468 18447 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:54.986685 19835 tablet_replica.cc:333] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:54.990551 18197 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 2 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.994959 19835 raft_consensus.cc:2243] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.996122 19835 raft_consensus.cc:2272] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:55.008970 19835 ts_tablet_manager.cc:1916] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:55.050745 18194 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.102958 19835 ts_tablet_manager.cc:1929] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:55.103698 19835 log.cc:1199] T d8bf6d66a261490caa69a80b24ea015f P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/d8bf6d66a261490caa69a80b24ea015f
I20260709 12:16:55.110323 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet d8bf6d66a261490caa69a80b24ea015f (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:55.264902 18369 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.266837 19804 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.279793 19038 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:55.281446 19796 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:55.315925 18441 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.317854 19564 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.371798 18298 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:55.401160 19038 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:55.423585 18293 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:55.444067 18300 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:55.454512 19038 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:16:55.456265 19804 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:16:55.465096 18371 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:55.466063 19798 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:55.472252 19796 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:55.474759 19040 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:55.482585 18446 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:16:55.484145 19564 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:16:55.515318 18447 raft_consensus.cc:2955] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:55.483328 19793 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 12:16:55.559365 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 74695.6us
I20260709 12:16:55.566493 19816 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:16:55.638960 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:55.680526 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 5 to 6, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:16:55.756150 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 248075us
W20260709 12:16:55.765931 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 247540us
I20260709 12:16:55.766697 19838 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:16:55.778088 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 75743.3us
I20260709 12:16:55.784493 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e2ee63a449648bfb064fadee177cadb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:55.806823 18441 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:55.902768 19044 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:55.904194 18298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:55.905031 18298 raft_consensus.cc:606] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:55.916174 19797 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:55.942373 18447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:55.942991 18447 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:55.939795 19797 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:55.956529 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:55.957157 18366 raft_consensus.cc:606] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:55.960551 18300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:55.961314 18300 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:55.965409 19039 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:55.966591 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:55.972016 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:55.972793 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:55.992995 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e2ee63a449648bfb064fadee177cadb with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:55.999395 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 353cbd244a354830a4e903a3b1be2b89 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 7)
I20260709 12:16:56.010774 18369 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:56.012285 19862 tablet_replica.cc:333] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:56.013410 19862 raft_consensus.cc:2243] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:56.016515 19839 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:56.019852 19862 raft_consensus.cc:2272] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:56.024036 18196 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:56.028707 18369 raft_consensus.cc:2955] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:56.036782 19808 raft_consensus.cc:993] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:56.037245 19804 raft_consensus.cc:1081] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:56.039199 19044 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59426
I20260709 12:16:56.039743 19044 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:56.040066 19044 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:56.023048 19862 ts_tablet_manager.cc:1916] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:56.077474 19862 ts_tablet_manager.cc:1929] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:16:56.078011 19862 log.cc:1199] T 9e2ee63a449648bfb064fadee177cadb P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/9e2ee63a449648bfb064fadee177cadb
I20260709 12:16:56.080447 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 9e2ee63a449648bfb064fadee177cadb (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:56.063086 19044 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:56.084018 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:56.084728 18300 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:56.085181 18300 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:56.086072 18300 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:56.087177 19044 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:56.087307 18300 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:56.088231 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet 353cbd244a354830a4e903a3b1be2b89 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:56.088444 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:56.089044 18441 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:56.090066 19863 tablet_replica.cc:333] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:56.091194 19863 raft_consensus.cc:2243] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:56.096230 18300 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:16:56.097827 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:56.098739 19774 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:56.099185 19774 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:16:56.100028 19774 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:56.110858 18441 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:16:56.092044 19863 raft_consensus.cc:2272] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:56.120009 19863 ts_tablet_manager.cc:1916] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:56.120724 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:56.154464 19863 ts_tablet_manager.cc:1929] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:16:56.155028 19863 log.cc:1199] T 353cbd244a354830a4e903a3b1be2b89 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/353cbd244a354830a4e903a3b1be2b89
I20260709 12:16:56.158056 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet 353cbd244a354830a4e903a3b1be2b89 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:56.209189 19797 raft_consensus.cc:993] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:56.209757 19797 raft_consensus.cc:1081] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:56.211867 18371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:46418
I20260709 12:16:56.212417 18371 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:56.212755 18371 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:56.255343 18371 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.258625 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:56.259286 18300 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:56.275882 18371 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:56.278326 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:56.278899 18441 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:56.281616 19808 raft_consensus.cc:993] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:56.282141 19808 raft_consensus.cc:1081] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:56.279186 18441 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:56.286012 18441 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.287340 18441 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:56.291785 18447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:16:56.292383 18447 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:56.292701 18447 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.273330 18300 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:56.334546 18441 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:56.335556 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:56.336863 19725 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:56.337412 19725 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:56.338403 19725 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.313683 18447 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.359696 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:56.360404 18300 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:56.360697 18300 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:56.361512 18300 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.362787 18300 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.369455 18194 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.374964 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:56.376425 18371 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.389899 18371 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:56.393182 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:16:56.395419 19847 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:56.396011 19847 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:56.397118 19847 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.375348 18447 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:56.401942 18300 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:56.430160 18194 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:56.439720 19840 raft_consensus.cc:993] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:56.440196 19840 raft_consensus.cc:1081] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:56.444340 18441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:56.444931 18441 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:56.445319 18441 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.476313 18441 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.485052 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:56.485996 18300 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.494055 18300 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:56.496019 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:56.497023 18371 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:56.497323 18371 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:56.498097 18371 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.499217 18371 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:56.496490 18441 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:56.515254 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:56.516216 19847 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:56.516786 19847 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:56.518606 19847 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:56.524060 18371 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:16:56.537614 18194 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:56.698005 18300 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:16:56.699887 19831 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:16:56.700948 18441 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:16:56.702332 19831 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:16:56.735435 18441 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:56.736945 19725 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:56.814175 18293 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:56.816059 19840 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:56.898202 18293 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:56.901903 19847 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:56.948896 18366 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:56.953825 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76479.8us
I20260709 12:16:56.958899 19849 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:57.119704 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:57.158169 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 151869us
I20260709 12:16:57.207702 18371 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:57.227417 19849 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:57.240885 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:57.241490 18366 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:57.243839 18300 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:57.246941 19881 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.247794 18446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8903d70080d24236be955433a92791c7"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:57.248597 18446 raft_consensus.cc:606] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:57.249854 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:57.253518 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:57.260670 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:57.331558 19847 raft_consensus.cc:993] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:57.332096 19847 raft_consensus.cc:1081] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:16:57.335378 18300 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8903d70080d24236be955433a92791c7"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:16:57.335899 18300 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:57.336228 18300 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:57.343433 18300 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.347105 18298 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:57.357999 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:57.358331 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:57.358572 18366 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:57.358860 18447 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:57.359166 18447 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:57.359854 18447 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.360867 18447 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:57.367148 18447 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:57.368391 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:57.369257 19798 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:57.369768 19798 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:57.370675 19798 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.372093 18300 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:57.379989 18366 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:16:57.397485 18447 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.399451 19813 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.400767 18196 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:57.404354 18263 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57dbd4858e8f41a2af67d6de6eae9928. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.405412 18366 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.412823 19798 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.412262 19808 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.416954 18446 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.442612 18366 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.447919 18298 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:57.452492 18198 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:16:57.494689 18263 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e62e9fad92b449d9315da4aa77082b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.495982 18446 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.497500 19813 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.502208 18371 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.506414 19813 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:57.540910 19808 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.542598 18366 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.550927 18446 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.558314 18447 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:57.582702 18366 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:16:57.583482 18412 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc5742413fd242b88ca8c4668e6b8144. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.584092 18300 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:57.584506 19880 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.585927 19847 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.596458 19847 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.603806 18366 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.613010 19724 raft_consensus.cc:993] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:57.613713 19724 raft_consensus.cc:1081] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:16:57.616043 18447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:16:57.616573 18447 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:57.616932 18447 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:57.624305 18447 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.634626 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:57.635422 18371 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:57.635895 18371 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:57.640240 18447 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:57.640995 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:57.641590 18298 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:57.638917 18371 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.659883 18371 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:57.667500 18300 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.672283 18298 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:16:57.673828 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:57.674963 19847 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:57.675674 19847 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:57.676594 19847 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:57.697781 18371 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:16:57.704980 18447 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:57.759775 18371 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.761155 19847 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:57.763624 18412 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc5f8db079f74a52b906175ad2c4cb56. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.764652 18300 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.767272 19880 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.795176 18196 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.803546 18196 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index -1 to 3, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.834216 18198 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:57.839053 19838 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.849280 18366 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.864264 18300 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:16:57.880501 18198 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index -1 to 2, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.887393 18446 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:57.888911 19813 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.925603 18366 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:57.929452 19903 ts_tablet_manager.cc:933] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:57.930261 19808 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.956230 19903 tablet_copy_client.cc:323] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:57.959415 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 8e62e9fad92b449d9315da4aa77082b4 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:57.959964 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 8e62e9fad92b449d9315da4aa77082b4 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-8e62e9fad92b449d9315da4aa77082b4
I20260709 12:16:57.980278 18310 tablet_copy_source_session.cc:215] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:57.992555 19908 ts_tablet_manager.cc:933] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:16:57.996788 19903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e62e9fad92b449d9315da4aa77082b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.006268 19908 tablet_copy_client.cc:323] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:16:58.007925 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 57dbd4858e8f41a2af67d6de6eae9928 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:16:58.008599 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 57dbd4858e8f41a2af67d6de6eae9928 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-57dbd4858e8f41a2af67d6de6eae9928
I20260709 12:16:58.020277 18310 tablet_copy_source_session.cc:215] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:58.023288 19908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57dbd4858e8f41a2af67d6de6eae9928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.052435 19903 tablet_copy_client.cc:806] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.053093 19903 tablet_copy_client.cc:670] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.061200 19903 tablet_copy_client.cc:538] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.080678 19903 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:58.086496 19908 tablet_copy_client.cc:806] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.089517 18300 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:58.091015 19849 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:58.088042 19908 tablet_copy_client.cc:670] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.106309 19908 tablet_copy_client.cc:538] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.109187 18369 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:58.110615 19849 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:58.153817 19908 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:58.197746 19903 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.198812 19903 tablet_bootstrap.cc:492] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:58.200861 19903 ts_tablet_manager.cc:1403] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.120s	user 0.022s	sys 0.006s
I20260709 12:16:58.204294 19903 raft_consensus.cc:359] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.206431 19903 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:58.207612 19903 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.221213 19908 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.222543 19908 tablet_bootstrap.cc:492] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:58.221601 19903 ts_tablet_manager.cc:1434] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 12:16:58.227201 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8e62e9fad92b449d9315da4aa77082b4 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:58.227924 19908 ts_tablet_manager.cc:1403] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.009s
I20260709 12:16:58.228559 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8e62e9fad92b449d9315da4aa77082b4 on tablet 8e62e9fad92b449d9315da4aa77082b4 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:58.239029 19903 ts_tablet_manager.cc:933] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:58.243621 19915 ts_tablet_manager.cc:933] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:58.247328 19903 tablet_copy_client.cc:323] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:58.249931 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet cc5f8db079f74a52b906175ad2c4cb56 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:58.250481 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet cc5f8db079f74a52b906175ad2c4cb56 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-cc5f8db079f74a52b906175ad2c4cb56
I20260709 12:16:58.239185 19908 raft_consensus.cc:359] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.255426 19908 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:58.256148 19908 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.258121 19915 tablet_copy_client.cc:323] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:16:58.259871 18458 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet dc5742413fd242b88ca8c4668e6b8144 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:16:58.260394 18458 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet dc5742413fd242b88ca8c4668e6b8144 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-dc5742413fd242b88ca8c4668e6b8144
I20260709 12:16:58.260603 19908 ts_tablet_manager.cc:1434] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:16:58.271656 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-57dbd4858e8f41a2af67d6de6eae9928 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:16:58.272058 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-57dbd4858e8f41a2af67d6de6eae9928 on tablet 57dbd4858e8f41a2af67d6de6eae9928 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:58.316259 18458 tablet_copy_source_session.cc:215] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:58.326011 19038 raft_consensus.cc:1217] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.334641 18459 tablet_copy_source_session.cc:215] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:58.347841 19915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5742413fd242b88ca8c4668e6b8144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.352216 19903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc5f8db079f74a52b906175ad2c4cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.359930 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:58.364027 19038 raft_consensus.cc:1217] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.378299 19903 tablet_copy_client.cc:806] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.378906 19903 tablet_copy_client.cc:670] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.391757 19903 tablet_copy_client.cc:538] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.395895 19915 tablet_copy_client.cc:806] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.396636 19915 tablet_copy_client.cc:670] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.412926 19915 tablet_copy_client.cc:538] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.427603 19903 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:58.466325 19915 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:16:58.490181 19903 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.491534 19903 tablet_bootstrap.cc:492] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:58.492625 19903 ts_tablet_manager.cc:1403] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.011s
I20260709 12:16:58.498980 18300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:16:58.499646 18300 raft_consensus.cc:606] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:58.497277 19903 raft_consensus.cc:359] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.502861 19903 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:58.504601 18446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:58.505198 18446 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:16:58.507745 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:58.509421 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:58.510164 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:58.503741 19903 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.515059 19903 ts_tablet_manager.cc:1434] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.006s
I20260709 12:16:58.519657 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-cc5f8db079f74a52b906175ad2c4cb56 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:58.520080 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-cc5f8db079f74a52b906175ad2c4cb56 on tablet cc5f8db079f74a52b906175ad2c4cb56 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:58.578822 19915 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.592782 19915 tablet_bootstrap.cc:492] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:16:58.599831 19915 ts_tablet_manager.cc:1403] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.165s	user 0.032s	sys 0.017s
I20260709 12:16:58.603653 19915 raft_consensus.cc:359] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.604676 19915 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:16:58.605710 19915 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:16:58.622231 19915 ts_tablet_manager.cc:1434] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:16:58.634399 19838 raft_consensus.cc:993] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:58.634961 19838 raft_consensus.cc:1081] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:58.637115 18366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:46418
I20260709 12:16:58.637662 18366 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:58.638026 18366 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:58.689392 18366 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:58.692628 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-dc5742413fd242b88ca8c4668e6b8144 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:16:58.693133 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-dc5742413fd242b88ca8c4668e6b8144 on tablet dc5742413fd242b88ca8c4668e6b8144 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:58.694554 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:58.695204 18293 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:58.705747 18366 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:16:58.706041 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:16:58.706666 18448 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:58.706990 18448 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:16:58.707831 18448 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:16:58.708683 18448 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:58.712786 18293 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:16:58.719370 18448 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:16:58.738751 19038 raft_consensus.cc:1217] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.746304 19798 raft_consensus.cc:1064] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:58.749131 19798 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:58.775938 19813 raft_consensus.cc:993] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:58.776573 19813 raft_consensus.cc:1081] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:16:58.778793 18369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:16:58.779363 18369 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:58.779698 18369 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.788828 19044 raft_consensus.cc:1217] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:16:58.726639 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:58.792290 19725 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:58.792740 19725 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:58.793417 19725 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:16:58.795713 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69166.2us
W20260709 12:16:58.841346 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 112964us
I20260709 12:16:58.864431 18198 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 5, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 5 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.865963 19813 raft_consensus.cc:1064] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:58.868884 19813 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:58.889125 19038 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.890718 19921 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.893685 18369 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:58.896296 18369 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:16:58.924347 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:16:58.953805 19038 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.955462 19044 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.957899 19921 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.961058 19038 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:58.965515 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:58.966109 18293 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:58.966439 18293 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:16:58.967180 18293 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:58.968328 18293 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.949836 18366 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.937476 18446 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.974488 18293 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:16:58.993854 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:16:58.994758 19923 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:58.995365 19923 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:58.995843 18369 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.996470 19923 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:58.996896 18446 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.002113 19813 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.015998 19921 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:16:59.019142 19904 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:59.029762 19813 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.059016 19904 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.061357 18441 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.073010 19039 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.086712 18369 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.092963 19813 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.094573 18372 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.099764 18197 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.103273 18446 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.124084 18196 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.142416 19039 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.145463 18194 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.177050 18293 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.177054 18300 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.185545 19039 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.187016 19813 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.196775 18372 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.196265 19904 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.202018 19905 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.208320 19838 raft_consensus.cc:1064] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:59.212531 18372 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.212394 19044 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.214152 19838 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.214815 19038 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.216655 19813 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.221570 18293 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:59.223282 19923 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:59.223417 19936 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.234895 19921 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.240317 18369 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.240873 18372 raft_consensus.cc:2955] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.269761 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e62e9fad92b449d9315da4aa77082b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.274091 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 57dbd4858e8f41a2af67d6de6eae9928 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.272835 19039 raft_consensus.cc:2955] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.286831 18446 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:59.289623 19924 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:59.317379 18194 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.333860 18194 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 3 to 4, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 1 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.351876 18369 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.354058 19044 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.354406 19880 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.369181 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e62e9fad92b449d9315da4aa77082b4 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:59.373705 18300 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.369182 18436 tablet_service.cc:1558] Processing DeleteTablet for tablet 57dbd4858e8f41a2af67d6de6eae9928 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54880
I20260709 12:16:59.374637 19880 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.375849 19945 tablet_replica.cc:333] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:59.376277 19847 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.377326 19945 raft_consensus.cc:2243] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.378214 19945 raft_consensus.cc:2272] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.402998 19838 raft_consensus.cc:1064] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:16:59.405678 19838 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.408993 19945 ts_tablet_manager.cc:1916] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.442997 18293 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:16:59.455597 19938 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 12:16:59.459293 19924 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:16:59.506202 19044 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:59.507983 19952 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.528193 19955 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.541030 19038 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:16:59.540583 19044 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.534972 19945 ts_tablet_manager.cc:1929] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.544126 19945 log.cc:1199] T 8e62e9fad92b449d9315da4aa77082b4 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/8e62e9fad92b449d9315da4aa77082b4
I20260709 12:16:59.546128 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 8e62e9fad92b449d9315da4aa77082b4 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:59.546334 19945 tablet_replica.cc:333] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:16:59.547441 19945 raft_consensus.cc:2243] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.549590 18296 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:59.551205 19958 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.552476 19945 raft_consensus.cc:2272] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.556038 19945 ts_tablet_manager.cc:1916] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.562798 19924 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:16:59.562244 18372 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.566601 18366 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:16:59.586460 18296 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.600409 19880 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.607673 18194 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 2 to 3, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.610232 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:59.639930 19945 ts_tablet_manager.cc:1929] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.640514 19945 log.cc:1199] T 57dbd4858e8f41a2af67d6de6eae9928 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/57dbd4858e8f41a2af67d6de6eae9928
I20260709 12:16:59.650106 19958 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.655426 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 57dbd4858e8f41a2af67d6de6eae9928 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:59.655045 18372 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.665565 18446 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.668175 18298 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.680294 19044 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.682230 19880 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.683382 18296 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.689417 19880 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.698740 19953 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.705905 19044 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.708271 18298 raft_consensus.cc:2955] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.730407 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc5f8db079f74a52b906175ad2c4cb56 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.731297 19039 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.754475 18194 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 1 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.764657 18194 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 3 to 4, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.805037 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet cc5f8db079f74a52b906175ad2c4cb56 with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46412
I20260709 12:16:59.814009 18446 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.823098 19039 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:59.823760 18367 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:16:59.825119 19958 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:59.826875 19958 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:59.836501 19880 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.839462 19963 tablet_replica.cc:333] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:16:59.841888 18372 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.845583 19963 raft_consensus.cc:2243] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.846349 19963 raft_consensus.cc:2272] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.843497 19039 raft_consensus.cc:2955] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.854919 19963 ts_tablet_manager.cc:1916] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.870157 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc5742413fd242b88ca8c4668e6b8144 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.878736 18194 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 4 to 5, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:16:59.894889 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet dc5742413fd242b88ca8c4668e6b8144 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42906
I20260709 12:16:59.925490 18372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:59.926065 19964 tablet_replica.cc:333] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:16:59.926188 18372 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:16:59.927199 19964 raft_consensus.cc:2243] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.927918 19964 raft_consensus.cc:2272] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.933300 18446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:50904
I20260709 12:16:59.936203 18446 raft_consensus.cc:606] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:16:59.932051 19963 ts_tablet_manager.cc:1929] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.943579 19963 log.cc:1199] T cc5f8db079f74a52b906175ad2c4cb56 P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/cc5f8db079f74a52b906175ad2c4cb56
I20260709 12:16:59.944438 18372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:16:59.944998 18372 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:16:59.945415 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet cc5f8db079f74a52b906175ad2c4cb56 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:59.939327 19964 ts_tablet_manager.cc:1916] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.947189 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:16:59.948952 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:59.949666 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:59.961760 19964 ts_tablet_manager.cc:1929] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:16:59.962544 19964 log.cc:1199] T dc5742413fd242b88ca8c4668e6b8144 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/dc5742413fd242b88ca8c4668e6b8144
I20260709 12:16:59.964200 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet dc5742413fd242b88ca8c4668e6b8144 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:16:59.969622 19923 raft_consensus.cc:993] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:59.970186 19923 raft_consensus.cc:1081] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:16:59.972368 19039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59430
I20260709 12:16:59.972923 19039 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:59.973250 19039 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:59.984342 19039 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.988111 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:16:59.988726 18372 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:59.987157 19039 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:16:59.988989 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:16:59.989612 18298 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:59.989008 18372 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:16:59.991669 18372 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:16:59.992914 18372 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:59.997954 18298 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:16:59.999334 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:00.000711 19873 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:00.001233 19873 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:00.002171 19873 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:00.006896 18372 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:00.040060 18197 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 5, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 5 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:00.176674 19880 raft_consensus.cc:993] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:17:00.177151 19958 raft_consensus.cc:1081] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:17:00.178470 18296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:17:00.179008 18296 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:00.179365 18296 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:00.186358 18296 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:00.188690 18296 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:00.189313 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:00.189313 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:00.189945 18446 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:00.190054 18367 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:00.190337 18446 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:00.191098 18446 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:00.192315 18446 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:00.198016 18446 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:00.198043 18367 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:00.199476 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:00.200229 19904 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:00.200663 19904 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:00.201553 19904 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:00.228204 18197 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:01.018186 18340 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.006s	sys 0.007s
W20260709 12:17:01.018538 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.006s	sys 0.007s
I20260709 12:17:01.288355 19904 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.289278 19904 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.292088 19904 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.328110 19968 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.328984 19968 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.329877 19969 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.331290 19969 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.335619 19969 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.340442 19969 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:01.344731 19971 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.362407 19971 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.365716 19971 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.360414 19967 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.349393 19904 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.421401 19969 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.467082 19936 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.469992 19979 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:01.426352 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800312be0 after lost signal to thread 17160
I20260709 12:17:01.415647 19873 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.463853 19968 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:01.469156 19904 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.488531 19969 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:01.489553 19968 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.490180 19969 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.463956 19967 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.490195 19968 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.473189 19936 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:01.474737 19873 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.490890 19969 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.474509 19979 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.564595 19976 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:01.565450 19976 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.571885 19853 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.572633 19853 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.511924 19980 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.681862 19975 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.700875 19985 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.700745 19975 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.701599 19985 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.703469 19975 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.704154 19975 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.704267 19985 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.704694 19975 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.705960 19983 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.706568 19983 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.709095 19983 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:01.709671 19983 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.711687 19983 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.714118 19983 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.719321 19975 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.720218 19987 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.720896 19987 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.715878 19988 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.723529 19988 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.726117 19988 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.704929 19985 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.727859 19984 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.729020 19987 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.696184 19970 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.731333 19983 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.921212 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:01.742303 19975 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:01.745136 19985 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.030011 19969 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:02.030295 19977 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:01.901798 19987 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.141664 19987 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:02.068631 19974 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.739862 19980 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:01.742476 19988 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.069852 19904 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:01.764467 19986 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.773337 19989 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:01.731186 19993 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:01.761518 19981 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:01.776333 19994 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:02.032403 19936 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.261466 19904 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.299597 19992 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.143637 19969 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.187549 19968 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.263597 19997 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.273340 19904 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.260632 19978 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.089955 19970 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.342211 19970 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:02.300607 19969 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.130036 19975 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.778499 19984 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.352113 19991 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.264590 19967 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.121217 19983 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.300385 19992 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.261822 19988 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:01.761574 19990 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.343118 19989 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.351042 19979 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.264334 19974 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.382287 19975 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.382932 19975 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.383509 19975 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.383870 19974 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.384557 19974 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.385700 19982 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.385860 19975 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.386546 19975 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.386384 19982 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.276299 19993 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.421250 19983 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.422084 19983 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:02.422211 19993 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:02.158442 19977 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.352872 19994 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.385154 19974 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.265859 19986 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:02.477411 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.126s	sys 0.275s
W20260709 12:17:02.478006 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.126s	sys 0.275s
W20260709 12:17:02.471657 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.006s	sys 0.005s
W20260709 12:17:02.479681 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.006s	sys 0.005s
I20260709 12:17:02.344309 19981 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.482010 19981 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:02.362879 19991 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.484313 19991 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.356644 19978 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.486475 19978 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.493867 20003 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.494449 20003 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.496623 20003 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:02.497368 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.006s	sys 0.010s
W20260709 12:17:02.497723 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.006s	sys 0.010s
I20260709 12:17:02.500658 20003 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.501212 20003 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:02.502266 18264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.013s	sys 0.004s
W20260709 12:17:02.502605 18264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.013s	sys 0.004s
W20260709 12:17:02.502633 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.005s	sys 0.011s
W20260709 12:17:02.503342 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.006s	sys 0.011s
I20260709 12:17:02.357148 19997 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:02.502677 18340 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.003s	sys 0.013s
W20260709 12:17:02.504253 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.003s	sys 0.013s
I20260709 12:17:02.426717 19969 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.477435 19873 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.490787 19993 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.492449 19991 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:02.490206 19981 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.387167 19975 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.492265 19978 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.568387 19989 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:02.377218 19990 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.523890 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:02.507153 19987 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.505903 19979 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.574654 18291 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:02.451970 19983 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:02.513913 19969 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:02.530859 19873 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.569706 19985 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.574708 19904 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.507563 19999 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:02.578085 19989 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:02.572129 19991 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:03.031798 18341 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.000s	sys 0.014s
W20260709 12:17:03.033471 18341 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.000s	sys 0.015s
I20260709 12:17:02.573160 19993 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:03.028215 18338 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.500s (SENT) [suppressed 4 similar messages]
I20260709 12:17:02.576778 19981 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:02.576961 19978 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.038838 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.040577 18367 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:02.736320 19969 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.025043 20003 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:02.624684 19979 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.044091 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.026367 19958 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.045717 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.027468 19980 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.049145 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.049762 18372 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:02.748872 19873 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.028705 19997 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.037211 19988 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.060562 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.061141 18291 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.086956 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.087654 18367 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.033179 19989 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.050527 20003 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.053318 20002 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.095310 20000 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.061365 19988 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.100700 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.101495 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.102550 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.102782 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.092232 19984 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.104395 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.105160 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.053403 19995 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.029844 19999 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.118110 18293 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:03.109872 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.111228 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.096028 19978 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.091658 19948 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.103617 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.139685 18296 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:03.119180 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.141315 18294 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.125223 19969 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.125880 19975 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.127825 19873 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.127738 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.123103 18299 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.137296 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.133013 18300 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:03.108661 19974 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.158630 19948 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.161775 20004 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.151896 19983 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.164207 19999 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.155347 19995 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.165820 19042 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:03.149643 19986 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.151584 19994 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.134440 19904 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.192020 19904 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.194518 19904 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.138697 19992 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.154788 19993 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.115829 19988 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.197698 19988 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.152837 19990 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.176705 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.200937 18372 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.177668 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.173424 19948 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.055608 19958 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.115046 20002 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.107169 20003 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.117630 20000 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.144662 19982 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.144898 19977 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.190472 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.151154 19979 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.146979 19978 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.153988 19991 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.149017 19981 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.157079 19989 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.188521 20004 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.207849 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.206182 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.260486 18372 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.223658 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.224144 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.265089 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.271447 18299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:50272
I20260709 12:17:03.272224 20000 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.272260 18299 raft_consensus.cc:606] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:03.204573 19995 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.225580 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.275605 19036 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:03.221160 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.278824 19978 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.281976 19978 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.274526 19979 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.303615 19979 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.306205 19979 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.284849 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.307929 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.309119 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.310107 19999 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.310670 19999 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.311194 19999 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.314162 19986 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.314821 19986 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.317171 19986 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.318094 19988 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:03.318627 19988 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.318675 19986 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.319190 19988 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:03.319625 19986 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.293907 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.321978 18366 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:03.294842 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.323341 18367 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.323647 19986 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.298452 20004 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.300495 19958 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.320596 20012 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.333186 20012 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.333691 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.334553 20004 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.323968 19978 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.334941 20004 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.291932 20003 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.296059 20002 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.331770 19989 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.336098 19989 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.335400 20004 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.337595 20012 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.339282 19989 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.335141 19978 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.342320 19978 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:03.345938 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.721s	user 0.011s	sys 0.049s
W20260709 12:17:03.346524 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.721s	user 0.011s	sys 0.049s
I20260709 12:17:03.350298 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.355014 19904 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.355630 19904 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.356057 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 244) took 3041 ms. Trace:
I20260709 12:17:03.356455 18446 rpcz_store.cc:276] 0709 12:17:00.315248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:03.355346 (+3040098us) service_pool.cc:224] Handling call
0709 12:17:03.355606 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:03.355761 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:03.355785 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:03.355840 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:03.355878 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:03.355923 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:03.356011 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:03.357748 19904 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.359926 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.321111 19995 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.363030 18197 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 287) took 2118 ms. Trace:
I20260709 12:17:03.364708 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.367815 19853 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.388788 18197 rpcz_store.cc:276] 0709 12:17:01.244296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:03.360368 (+2116072us) service_pool.cc:224] Handling call
0709 12:17:03.362966 (+  2598us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":0}
W20260709 12:17:03.392534 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.030s	sys 0.091s
W20260709 12:17:03.392843 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.030s	sys 0.091s
I20260709 12:17:03.370153 19978 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.392371 19968 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.372483 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.406361 18448 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.386904 18441 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.399166 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:03.407842 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800339e20 after lost signal to thread 18232
W20260709 12:17:03.409143 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002477c0 after lost signal to thread 18324
W20260709 12:17:03.410178 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800205920 after lost signal to thread 18399
I20260709 12:17:03.388962 18446 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
W20260709 12:17:03.393867 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 365646us
I20260709 12:17:03.371173 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.411171 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.411422 18447 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.403152 20007 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:03.411840 18444 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.412442 19987 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.412276 20007 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.412953 19987 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.411850 19968 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.391919 19976 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.414492 20007 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.415093 20007 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.403775 19853 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.415657 20007 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.416015 19880 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.416616 19880 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.400385 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.417789 20007 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.418445 19880 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.415855 19853 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.416141 19989 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.419474 19989 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:03.420210 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800308320 after lost signal to thread 18473
W20260709 12:17:03.421315 19065 debug-util.cc:398] Leaking SignalData structure 0x7b080024c9a0 after lost signal to thread 19068
I20260709 12:17:03.421531 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.421756 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.422226 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.336505 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.424943 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.426316 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.425549 19041 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.397667 19904 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.444051 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.446614 18367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:03.447170 18367 raft_consensus.cc:606] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:03.448014 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.448339 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.446656 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.449112 18300 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.415030 19987 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:03.451411 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.040s	sys 0.091s
I20260709 12:17:03.451221 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.452560 18294 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.439675 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:03.453510 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.040s	sys 0.091s
I20260709 12:17:03.453634 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.455750 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.421320 19989 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.461800 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.392969 19995 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.463029 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.462404 18296 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:03.465839 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.472559 19958 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.473093 19958 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.475337 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.455899 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.458132 19991 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:03.452692 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 114052us
I20260709 12:17:03.492704 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.493577 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.493876 19999 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.452481 19982 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.458832 19873 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.467904 20013 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.498867 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.500172 19978 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.479756 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.480554 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.481510 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.482185 20015 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:03.481361 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.504025 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.483717 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.484747 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.466209 19968 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.450796 19904 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.486773 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.487581 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.487949 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.488430 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.488555 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.471729 19989 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.490785 19853 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.491583 19995 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.520246 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.496740 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.495399 19831 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:03.463527 19987 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.497455 19951 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.497783 19979 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.495509 19991 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.498839 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.498016 19873 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.479426 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.451278 20004 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.500461 18444 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.500252 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.501189 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.506332 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.508601 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.482268 19958 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.503773 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.510422 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.538370 18371 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:03.538475 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.538578 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.539083 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.539690 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:03.540381 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.540688 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.541522 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.541754 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.542608 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.511729 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:03.511801 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.521551 20015 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.517402 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.518275 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.545040 20015 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.546079 18441 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:03.515456 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.548152 18366 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:17:03.549795 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.524268 19991 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.550406 18300 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:03.515964 19989 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.553181 19989 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.520870 18367 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:03.519649 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.511005 19993 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:03.523347 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.532267 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.535902 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:03.518657 18299 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:03.510995 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.571995 18372 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:03.508112 19982 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.542711 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.513806 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.523025 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.500236 20013 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:03.590281 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73787.8us
I20260709 12:17:03.569906 18449 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:17:03.521927 19831 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.483011 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.572358 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.546082 20014 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.577003 18370 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:03.579628 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.535631 19873 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.524121 19979 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:03.521714 19987 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.582232 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.583128 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.583526 19904 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.584184 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:03.605155 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:03.585611 20004 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.531805 19988 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.534584 19951 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.587383 19038 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:03.587217 19999 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:03.589524 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.610351 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:03.595400 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:03.597738 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.598083 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.599632 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:03.582935 19991 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.601855 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:03.648494 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49596.3us
I20260709 12:17:03.600646 19044 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:03.608237 18367 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:03.605647 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:03.593521 19831 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:03.594209 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:03.656086 19991 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.656335 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.658078 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.658910 18371 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:03.609512 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80614.7us
I20260709 12:17:03.650622 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:03.651636 19971 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.560539 19993 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.600317 19987 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.663815 19039 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:03.664201 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.665144 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.661935 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:03.675971 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 70422.8us
I20260709 12:17:03.677383 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:03.610445 19951 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.731698 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.603639 20014 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.744215 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:03.744868 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:03.611367 19968 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:03.662142 20004 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.755856 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.672868 19993 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:03.760002 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.762050 19043 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:03.764775 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:03.663714 19971 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.664281 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.770768 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:03.776590 19988 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.753520 19978 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.777248 19988 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:03.771807 20003 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.748541 19972 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:03.752902 20018 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.804733 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.784390 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.751585 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.751861 19923 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.822641 20014 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.757334 19968 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.847777 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:03.849529 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.823552 19923 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.851132 19923 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.851586 19979 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:03.851729 19968 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:03.853163 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 86182.9us
I20260709 12:17:03.801350 19978 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.773499 19990 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.857805 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:03.779477 19976 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:03.772060 19971 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.759557 20004 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.818780 20018 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:03.867220 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:03.804571 20003 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:03.870497 18371 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:03.880753 20004 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.855966 19853 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:03.862454 19978 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.879374 20003 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.870612 19976 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.862141 19990 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.875531 20018 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.919152 19853 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:03.926030 19976 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:03.926578 19990 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.939181 19982 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.933651 19853 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:03.938526 20013 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:03.949491 19987 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.954823 20003 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:03.955585 20004 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:03.985894 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 326542us
W20260709 12:17:03.986829 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 321419us
I20260709 12:17:03.991272 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:03.994225 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:03.996999 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:03.999791 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:04.003929 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 233083us
I20260709 12:17:04.005402 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:04.008255 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.012542 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.014124 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.016113 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.016391 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.017611 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.014215 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:04.020606 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.020520 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.020733 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.023568 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.023777 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.025851 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.026490 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.027815 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.028239 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.022444 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.025893 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.019212 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.038117 19035 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.037911 19972 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.038745 19972 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.039423 19972 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.042172 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.046355 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.047746 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.048347 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.019249 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.049820 20003 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.061235 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.062291 18449 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:04.063364 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.065904 19991 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.063472 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.070631 18371 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
W20260709 12:17:04.077545 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59129.2us
I20260709 12:17:04.079264 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.083209 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:04.086514 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 78638.6us
I20260709 12:17:04.088284 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.089526 19991 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.090148 19991 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:04.090735 19991 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.090765 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:04.091598 20021 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.093000 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.094744 19977 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.100188 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:04.101473 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.105216 18294 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:04.106956 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.107491 20015 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:04.108937 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:04.109102 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.095212 19977 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.127961 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.129627 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.131951 18369 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.132359 20004 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:04.134465 19873 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.006s
I20260709 12:17:04.135823 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.135274 19873 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.139001 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:04.140427 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 290982us
I20260709 12:17:04.144448 20013 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.145280 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.147301 19982 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:17:04.128197 19977 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.146966 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.124461 20030 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.154014 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.154214 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.154976 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.156217 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.161630 20020 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:04.162292 19982 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.163700 19982 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.164289 20004 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:04.164736 20004 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:04.165125 20004 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.164976 19982 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.166745 19982 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.144476 19987 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.171969 20002 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.159755 20030 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.173444 20030 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.184334 19039 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.185145 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:04.187932 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.188948 18296 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:04.185527 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.190212 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.191125 19975 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:04.192664 20002 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:04.193172 20002 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:04.193636 20002 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.193699 19975 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:04.197539 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.198781 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.199566 18299 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:04.196202 19975 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.203933 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:04.200614 19968 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.212345 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:04.214457 20014 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:04.219648 20014 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.215878 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.221012 19971 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.222077 20014 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.224114 19923 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.225579 19951 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.226475 19993 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.228369 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.228551 19038 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.229822 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.230126 20018 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.230638 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.230785 20003 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:04.231062 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.232151 19991 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.245149 20003 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:04.245587 20003 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.245713 18371 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.232911 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:04.230769 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.233747 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:04.233711 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55007.6us
I20260709 12:17:04.239485 20019 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:04.249480 20019 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.249852 20019 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.239880 20028 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:04.251161 19988 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:04.241350 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.242059 19904 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:04.243384 19990 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.232148 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.244375 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.252111 20016 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.253803 19988 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.254017 20016 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.254236 19988 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:04.215072 19968 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.254724 19904 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.257344 19983 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.259220 19873 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:04.259254 19904 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.257158 20016 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.261754 19904 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.262805 20016 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.234524 19923 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.257093 19988 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:04.265101 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51876.1us
I20260709 12:17:04.265230 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.266861 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.266322 19988 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.269764 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.269995 20028 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.271082 20028 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.271447 20028 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.272325 19990 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.263350 19904 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.272778 20028 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.276607 20018 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.281941 20003 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.284250 20028 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.288578 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.288911 18371 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.289175 19978 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.290547 19041 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.294916 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.296016 20014 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.296335 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.297225 20014 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.194437 19976 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.300626 18371 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.301059 19904 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.301496 19978 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.301776 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.302489 18369 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.302840 19853 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.304086 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.305141 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.306154 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:04.306461 19831 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.018s
I20260709 12:17:04.319700 19831 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.321064 19036 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.306823 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.307245 18366 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.306499 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.307408 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.308068 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.307531 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.324581 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.324756 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.325276 18366 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:04.325481 18367 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:04.327407 19831 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.251067 20019 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.311481 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.328442 18441 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:04.328263 20030 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.328763 20019 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.308640 19991 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.312831 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:04.313334 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55128.3us
I20260709 12:17:04.315570 20029 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:04.316819 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.317312 19971 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.307157 19972 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:04.311627 19951 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:04.329530 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.335075 18369 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.335896 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.326846 20004 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.337136 20029 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:04.352030 20029 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:04.341931 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.342655 20020 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:04.353241 20020 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.350821 19993 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.343473 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:04.355553 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.345920 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.347312 19972 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.353585 19853 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.344851 19041 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.345321 20017 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:04.359839 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.361811 20020 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.362385 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.361464 20017 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.365813 18369 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.365576 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.369122 20017 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.370555 20003 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.370468 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.372102 20014 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.372327 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.372607 20014 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.373716 20014 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.374054 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.375568 19976 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.375722 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.375839 19977 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.377442 18367 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.379679 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.381356 19977 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.384080 19991 raft_consensus.cc:993] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:17:04.339469 20002 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.387254 19991 raft_consensus.cc:1081] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:17:04.388718 20002 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:04.391055 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:04.392161 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.392094 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:04.392897 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.395084 19976 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.396016 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.396869 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.397583 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.398773 19991 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.399230 19977 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.392282 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.402366 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:04.402649 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.403141 19831 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:04.404098 19831 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.404860 20014 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.404901 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.405540 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.409937 20004 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.411957 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:04.415287 20030 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.415804 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.416087 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.420255 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.418586 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.421432 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.424957 18374 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.426723 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.429289 20028 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.429466 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:04.430804 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.433156 18374 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.426573 19976 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.436444 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.438936 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.439101 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.439581 20028 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.441920 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.442116 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.441401 19976 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.442601 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.443284 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.444358 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.445794 18370 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.446193 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.444463 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.447185 18371 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:04.447515 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.447384 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:04.448849 19995 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:04.449136 18373 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.449211 19995 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:04.449769 19995 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:04.443295 18366 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:04.450856 18297 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:04.428782 20004 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.451038 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.456108 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.458040 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.451468 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.462395 19989 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:04.462981 19880 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:04.464870 19958 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:04.465086 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.464871 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:04.472298 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.435408 20030 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.471465 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.469868 19978 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.475099 19976 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.478695 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:04.480501 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.471638 20016 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.481462 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.481904 19978 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.482174 19041 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:04.483175 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.484037 19978 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.467445 19995 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.486364 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.487102 19978 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:04.487279 19036 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.487615 18197 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 3 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:04.488639 19978 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.488881 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.489832 19036 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:04.494022 19036 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.494421 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.494628 19995 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.495191 19037 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:04.495033 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.497797 19041 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.501140 19041 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.502751 19904 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.503671 20016 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.503926 19904 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.504452 19904 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.505087 20016 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.505501 20016 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:04.505366 19904 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.515826 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.515718 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.518723 20040 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.519111 20040 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.520031 20040 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.522333 19904 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.522804 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.524139 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:04.516131 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50138.9us
W20260709 12:17:04.523855 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51658.9us
I20260709 12:17:04.526831 18447 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:04.527604 18293 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.523862 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.506525 20016 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.529554 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.529919 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.530723 20016 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.531165 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.532261 20019 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.532647 20019 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.532995 20019 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.534632 19038 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:04.470366 20029 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.537458 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.537930 18374 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:04.538295 19991 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.538570 20029 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:04.541638 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:04.524226 19976 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.543442 19853 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.543934 19853 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:04.544973 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84679.6us
I20260709 12:17:04.544921 19853 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.545455 19978 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:04.545746 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.546897 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.548276 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.549090 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:04.549994 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.550331 20014 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.561381 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.565310 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.565754 18293 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:04.566174 19995 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.566602 19995 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.570863 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.564095 20029 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.569377 19982 raft_consensus.cc:2764] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:04.583528 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64708.1us
W20260709 12:17:04.581853 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45592.4us
I20260709 12:17:04.573593 19977 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:04.591070 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44253.4us
I20260709 12:17:04.592234 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.592413 20040 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:04.599478 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51285.6us
I20260709 12:17:04.612690 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.600138 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.597684 19978 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.616410 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.619138 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.613698 20038 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:17:04.625535 20041 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.626003 20041 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.620549 19988 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.630697 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.631670 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.632076 19988 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.632437 19958 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:04.632830 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.633613 19958 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.634402 19831 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:04.634859 19831 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:04.634691 19989 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:04.635032 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.638518 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.636022 19831 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.641736 20041 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.642994 20041 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.623239 19982 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:04.649917 19880 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.650300 19880 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.645252 20030 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.623685 19995 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.647373 19982 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.659415 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.656570 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.651206 19880 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.666466 19880 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.662106 19982 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.651353 20038 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.655678 20003 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.660518 19995 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.675422 19971 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.643414 19831 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:04.681118 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.652451 20042 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.686120 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.628468 20019 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:04.684259 19991 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.684352 20016 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.691392 20016 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.691815 20016 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.693531 19971 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:17:04.702795 20014 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.703215 20014 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:04.703595 20014 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.709403 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.710800 19904 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:04.715716 19991 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.716694 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:04.716771 19991 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.717779 19991 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.718102 19991 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:04.719324 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.722625 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:04.724056 18296 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:04.724120 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.722622 19990 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:04.732235 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97252.7us
I20260709 12:17:04.734769 19904 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.766134 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.767563 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.729509 19972 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:04.771879 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52016.4us
I20260709 12:17:04.782050 19982 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.785650 19989 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:04.786731 19989 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.787045 19989 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:04.787142 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca383fa1243547c3b32dfbd6857a3fba. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.789496 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.790370 20042 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.790113 19989 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.771633 19990 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:04.791505 19989 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.788319 20014 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.791311 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.791805 19990 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.792462 20030 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.793660 20014 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.788714 20016 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.794814 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.796228 20016 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.790863 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.797171 20049 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.798460 19978 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:04.798498 20049 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:04.799010 19978 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:04.799721 20050 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.800105 20050 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.799891 19978 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.800383 19995 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:04.801192 20038 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:04.789191 20041 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.802335 20041 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.802425 20042 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.802716 20041 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.802870 20042 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.800932 20050 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.803923 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.804533 20050 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.803757 20042 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:04.801416 19958 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.763079 20039 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.817317 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:04.831665 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.837337 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.838464 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.837402 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.840895 18296 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:04.843199 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.844241 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.845374 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.846107 19993 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.846522 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.846720 19993 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:04.847072 19993 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:04.847393 19880 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.848595 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.849153 20019 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:04.787551 19991 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.816331 19958 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.852278 20042 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.847488 19993 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.857816 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.773512 19972 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:04.861747 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 142617us
I20260709 12:17:04.863629 18444 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:04.803730 20041 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.867246 20041 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.865837 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:04.857080 18449 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:17:04.871855 18449 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.872219 18449 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:04.856175 19977 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.873809 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.876168 18374 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:04.877821 19982 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:04.852712 19958 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.880177 19958 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.881848 19988 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:04.886684 19978 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:04.893111 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.864291 20039 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:04.897394 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:04.898839 18446 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:04.899158 19991 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.902338 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:04.899263 18449 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.917939 19982 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.869465 19972 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:04.937018 19972 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.937152 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.947353 20020 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.923957 20030 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:04.952898 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52549.2us
I20260709 12:17:04.955592 20038 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:04.957118 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:04.961880 19038 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:04.969743 18445 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:04.976174 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:04.976747 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:04.906602 20040 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.978328 18449 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:04.988379 20042 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:04.988806 20042 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:04.989627 20042 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.990880 20042 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.994033 18194 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 2 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:04.997972 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92242.5us
I20260709 12:17:04.972682 20039 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.999504 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.000558 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:05.003106 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:04.995456 19993 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.004827 20017 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.007200 20017 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:05.008109 20017 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.974552 19989 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:05.011799 20028 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.012355 20028 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:04.998057 20038 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:05.014237 20038 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:05.014611 20038 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:04.985351 19976 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:05.016625 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.018759 19041 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:05.020463 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.022085 20059 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:04.996287 20020 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.027098 18440 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:05.028378 19982 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:05.028810 19982 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.029192 19982 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:05.027341 19976 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.034000 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:05.032092 20053 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.034812 18449 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:05.036909 20020 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.037314 20039 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:05.037699 20039 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:05.037699 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.038715 20059 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.038729 20020 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.040441 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:05.041147 20059 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:05.041688 20059 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.042500 20059 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.035772 19990 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.052989 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.045674 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:05.045228 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.041074 20028 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.031136 19976 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.035705 20053 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.056087 19923 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:05.056223 20053 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.056933 19923 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.058000 19923 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:05.061087 18447 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:05.005554 19978 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:17:05.063555 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:05.064291 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.063760 20041 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:05.064893 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61843.4us
I20260709 12:17:05.061102 18442 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:05.067406 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:05.068876 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.072496 19990 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.075887 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.065068 20041 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.076726 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.082286 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.083447 19978 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.083954 19978 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:05.084349 19978 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.087324 19993 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.089313 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:05.089771 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.072593 20040 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:05.082283 20049 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:05.093093 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:05.095026 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.095894 18293 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:05.096872 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.097314 20053 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.097445 18297 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:05.097826 20053 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.097910 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.099833 20053 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.100301 19043 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:05.100455 19853 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:05.100972 19853 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.101866 19853 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.103011 18298 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.104130 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.105500 19043 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:05.093541 20040 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:05.091109 20028 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:05.106927 20040 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.107684 20028 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.108136 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.108697 18367 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:05.108980 18367 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.109555 18367 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:05.076870 20057 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:17:05.117604 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.117962 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:05.110142 20028 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.116143 18366 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:05.116425 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:05.120724 20049 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.122037 19982 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.124454 18298 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.125666 19038 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:05.128772 18367 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:05.130826 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.131783 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:05.132354 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.132678 19043 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.130844 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.134810 18369 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.135152 18298 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:05.138263 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.138988 18374 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.122895 20049 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.144399 19038 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.137492 19993 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:05.148651 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.150319 18298 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:05.150635 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.128476 20038 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.143605 19968 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:17:05.154481 19972 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:17:05.208117 18298 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:05.202351 18367 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:05.205561 18295 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:05.206995 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.209447 18366 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.210191 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.207609 20017 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:05.220647 20017 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.221721 19990 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:05.222601 19990 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.223801 19990 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.227558 20017 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.227687 19978 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.228650 18369 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:05.228590 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:05.229460 19038 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.230223 20063 ts_tablet_manager.cc:933] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.230621 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.231353 18369 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.237668 20041 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.239331 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.239681 20032 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:05.240060 20032 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.240078 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.241676 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:05.242352 19038 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.240792 18295 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:05.245841 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:05.247186 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.247375 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:05.248165 20020 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.248543 20020 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:05.248910 20020 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.246119 20038 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:05.250174 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.249349 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.254920 18300 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.256038 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.243677 19978 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:05.258405 20053 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:05.275462 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:05.266642 20064 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:17:05.267046 19982 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.272269 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:05.272907 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.272046 18194 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.276355 18296 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.280278 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.276684 18297 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.252516 20039 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:17:05.282233 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.286386 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.297617 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.283699 20029 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:05.298107 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.298508 20029 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.298723 18298 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.299582 18297 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:05.299387 20029 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.300885 20029 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.306869 18296 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.307458 18198 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.310559 18198 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.284700 20032 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.312382 18300 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.283852 20053 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:05.333668 20053 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.318149 20039 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.338516 20032 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.318888 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.320236 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.342886 18293 raft_consensus.cc:2376] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2. Re-sending same reply.
I20260709 12:17:05.322005 18292 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:05.322820 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.324035 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.345261 18367 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.325212 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.325773 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.335258 18198 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.324486 18196 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 4 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 5 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.327068 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:05.328068 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.348134 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.349188 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:05.349890 19988 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.350399 19988 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.351212 19988 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.351501 18198 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:05.354331 19983 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.355213 19983 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.356053 19983 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.357836 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.330293 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.331636 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:05.359416 20017 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.360204 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:05.363969 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.357209 19978 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.363754 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.365486 20028 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:05.366956 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.360682 19971 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.365533 20039 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:05.394198 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.365548 19982 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.398366 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.400226 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:05.401243 20038 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:05.403553 20028 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.375708 20049 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.369864 19978 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.397732 20039 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:05.411010 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 261115us
I20260709 12:17:05.412111 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.396684 20063 tablet_copy_client.cc:323] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:17:05.416446 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:05.423442 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.429360 20076 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.413247 20039 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.435297 18297 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:05.438439 19038 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:05.416882 19990 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.440341 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.441027 19037 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:05.443353 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.427404 20030 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:17:05.458467 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.459681 18366 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.459141 20042 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:05.440276 20020 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.473830 20042 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:05.474200 20042 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:05.475109 20042 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.454243 19978 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.457720 18297 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:05.435303 20049 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:05.439277 20076 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.483144 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.483330 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:05.489549 20041 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.489470 20076 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.490193 20041 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:05.490777 20041 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.491021 20049 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:05.491968 20049 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.462913 20040 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.487702 20019 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:17:05.505288 20019 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:05.505632 20019 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:05.506460 20019 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.481853 20017 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.498291 20028 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.498432 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:05.508561 20070 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.003s
I20260709 12:17:05.509433 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.509824 20017 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.510551 20017 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.501003 18440 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:05.488082 20038 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.512399 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:05.513748 20028 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.513871 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.515738 19982 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.517239 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:05.517729 19853 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:05.518214 19982 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.520001 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.520210 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.520714 18291 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.523056 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:05.524523 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:05.527002 18366 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:05.526979 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.527410 18194 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.532707 20050 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.534490 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.536470 19972 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:05.537649 19968 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:05.534160 20078 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.538040 19968 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:05.538998 19968 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.539669 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet ca383fa1243547c3b32dfbd6857a3fba from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:17:05.540275 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet ca383fa1243547c3b32dfbd6857a3fba from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-ca383fa1243547c3b32dfbd6857a3fba
I20260709 12:17:05.544209 20019 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:05.532840 20056 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:17:05.554046 20050 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.556875 20017 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.559674 20041 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.560099 20020 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:05.560889 18459 tablet_copy_source_session.cc:215] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.561676 20042 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:05.561741 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:05.562023 18447 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:05.564670 19972 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:05.565318 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.565282 20080 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.566039 20078 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.566824 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.567459 19968 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.574030 18296 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:05.561795 20041 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.574481 18373 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:05.574942 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:05.576525 20039 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.574950 18198 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 5 to 6, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 6 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.596976 19037 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:05.604461 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:05.618361 18198 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.622248 18291 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.629035 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.634408 20041 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.614419 20081 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.638505 18444 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:05.604043 20019 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:17:05.641137 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.644873 20077 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.636204 19990 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:05.649262 18296 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:05.633276 20030 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:05.654740 19990 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.648465 20053 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:17:05.647722 19831 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:05.665472 20083 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:17:05.665462 19831 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:05.669335 18444 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:05.669657 18296 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:05.651906 20077 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:05.676278 19990 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.677134 19037 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:05.701205 20077 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.677860 18447 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:05.678715 20039 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:05.678593 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:05.680823 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:05.682530 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:05.685722 19831 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.691642 18292 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:05.665823 20078 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.693588 18298 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:17:05.702605 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98212.3us
I20260709 12:17:05.703331 20039 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:17:05.726976 18293 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.727566 18300 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:05.729156 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.730214 20039 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.730660 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:05.731851 20050 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:05.732378 20050 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:05.733197 20050 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:05.743708 20040 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:17:05.744953 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 46549.8us
I20260709 12:17:05.697424 18196 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:05.746294 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45739.7us
W20260709 12:17:05.775204 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 155443us
I20260709 12:17:05.779598 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:05.783392 19038 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:05.749643 19853 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:05.750365 19990 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 12:17:05.837803 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52458.6us
W20260709 12:17:05.828385 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51763.9us
W20260709 12:17:05.841959 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:05.846622 18299 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:05.824962 20056 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:17:05.863931 18445 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:17:05.868161 20090 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:17:05.862720 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:05.825222 19993 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:17:05.890909 20102 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.893667 20053 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:05.930719 20059 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:05.886957 19923 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:17:05.937284 20050 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:05.933745 20059 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:05.942108 20100 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:17:05.944197 20063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca383fa1243547c3b32dfbd6857a3fba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.953431 18296 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.955353 19988 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:05.961228 18444 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:05.963490 20096 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:17:05.947098 20059 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:05.967192 20084 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.311s	user 0.000s	sys 0.006s
I20260709 12:17:05.970515 20107 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:17:05.973480 20102 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.945300 20050 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:05.968271 20081 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:17:06.009577 18449 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:06.013793 18292 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.015424 18299 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.016095 19988 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:06.016633 20050 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.018827 19988 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:06.023321 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:06.023934 18444 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:06.031065 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:06.038748 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:06.042030 18291 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:06.046923 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:06.050200 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:06.050890 19042 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:06.058008 18366 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.061372 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:06.082574 20098 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:17:06.083495 18373 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.084473 18374 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:06.089627 18293 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:17:06.103164 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 50487.8us
I20260709 12:17:06.087790 20041 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.130388 18369 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:06.133634 20079 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:17:06.090680 19972 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:17:06.115132 20102 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.111028 20040 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:17:06.147639 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:06.131477 20042 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.145361 20102 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.161249 20042 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.171079 18300 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.176067 20042 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.160965 20102 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.179174 18300 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:06.184490 18373 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.204638 18291 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:06.224998 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:06.225883 19042 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:06.233125 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:06.227913 19853 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:06.312465 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:06.314373 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 267507us
I20260709 12:17:06.326313 20081 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.326959 20081 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.240509 20049 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.237964 19880 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 12:17:06.359730 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:06.313804 19978 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.379400 19978 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.364161 19983 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:17:06.364528 20079 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.381910 19978 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:06.361414 20030 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.369443 18368 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.382074 20079 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.382711 20030 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.385021 20030 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.315171 19853 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.389036 19853 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.334620 20049 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.391152 20049 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.392925 18369 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:06.377848 20041 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.332501 20040 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.395377 18194 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:06.403373 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:06.404062 18367 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:06.393815 20040 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:06.414361 20040 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:06.415540 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:06.416181 18368 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
W20260709 12:17:06.417083 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52122.2us
I20260709 12:17:06.417742 20105 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.421461 18198 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.423208 18198 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.409775 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:06.428030 18198 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.434783 20081 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:06.440872 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:06.429184 18198 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.444749 18198 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.451753 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:06.452585 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48728.5us
I20260709 12:17:06.455303 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:06.384541 20079 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:06.455955 18449 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:06.363541 20063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59422 (request call id 384) took 1279 ms. Trace:
I20260709 12:17:06.482239 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:06.448503 20080 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.483345 18291 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:06.488613 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:06.458665 20081 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.458551 20078 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.497076 18291 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.500448 18293 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.497664 18449 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.456224 20102 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.502698 18440 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:06.453488 19982 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:17:06.487471 20080 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.527055 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:06.527751 18449 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:06.494501 19989 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:06.495728 20081 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.500054 20078 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.557977 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:06.529304 20080 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.553644 19989 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.556169 20081 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.560611 20078 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.565183 19990 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.610412 19989 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:06.630264 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.033s	sys 0.062s
W20260709 12:17:06.630613 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.033s	sys 0.062s
I20260709 12:17:06.610740 19977 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.636128 18449 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:06.643394 18298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.027s	sys 0.073s
W20260709 12:17:06.643770 18298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.027s	sys 0.073s
I20260709 12:17:06.628892 19990 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:06.710536 19990 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.710870 20042 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.639932 20040 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
W20260709 12:17:06.741077 20120 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:06.707396 19977 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.760411 19990 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.762596 19990 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.483762 20063 rpcz_store.cc:276] 0709 12:17:05.049211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.056771 (+  7560us) service_pool.cc:224] Handling call
0709 12:17:05.230547 (+173776us) ts_tablet_manager.cc:934] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
0709 12:17:05.832918 (+602371us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:17:06.123823 (+290905us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:06.132529 (+  8706us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:17:06.318429 (+185900us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:17:06.319359 (+   930us) tablet_replica.cc:206] Creating consensus instance
0709 12:17:06.328485 (+  9126us) ts_tablet_manager.cc:989] Replica registered.
0709 12:17:06.328855 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":29188,"tablet-copy.queue_time_us":92088,"thread_start_us":163176,"threads_started":2}
I20260709 12:17:06.765202 20063 tablet_copy_client.cc:806] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:06.765426 20080 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.765949 20063 tablet_copy_client.cc:670] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:06.767731 20080 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:06.768666 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:06.769524 18373 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.793711 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:06.794668 18449 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
W20260709 12:17:06.795331 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:06.755714 20042 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:06.825500 20127 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.828002 20127 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.830711 20127 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:06.831614 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 345674us
I20260709 12:17:06.940984 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:06.924160 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:06.941768 18446 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.942009 18292 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:06.918336 18449 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.951402 18299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.021s	sys 0.058s
W20260709 12:17:06.951905 18299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.021s	sys 0.058s
I20260709 12:17:06.957024 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:06.834723 20127 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.958261 20127 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:06.959060 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:06.958951 20081 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.959667 18442 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.960534 20127 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:06.926149 20134 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:06.961043 20081 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:06.961457 20134 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.930752 20102 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:06.963188 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:06.963974 18374 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.894251 20078 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:06.970204 18442 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:06.991957 19831 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:06.883522 20128 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:17:06.999667 19993 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:07.000928 20126 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.282s	user 0.007s	sys 0.000s
I20260709 12:17:06.892092 19989 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:06.997617 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:06.826455 19977 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.015305 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:07.019933 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:06.986838 20135 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.927508 20019 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:06.966182 20079 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.988112 20028 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:06.887069 20049 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.216897 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:07.087771 18374 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:07.072813 18373 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:07.084200 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:07.229130 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:07.214393 18443 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.243569 18447 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:07.003847 20039 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:07.245280 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:07.246789 18369 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:07.241914 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 272881us
I20260709 12:17:07.210441 20078 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:06.999006 19978 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.241s
I20260709 12:17:07.021337 20059 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:07.052137 19972 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.083877 20042 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.115137 19982 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:07.277842 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 284727us
I20260709 12:17:07.268975 18448 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:06.997578 19983 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.255s
I20260709 12:17:07.247516 20134 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.318475 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1824) took 1280 ms. Trace:
I20260709 12:17:07.244530 20129 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:07.022269 19993 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.303450 19977 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:07.243105 20079 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.243301 20028 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.242744 20135 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.320358 18444 rpcz_store.cc:276] 0709 12:17:06.031031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.031144 (+   113us) service_pool.cc:224] Handling call
0709 12:17:06.031431 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:06.031720 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:06.031772 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:06.042994 (+ 11222us) log.cc:844] Serialized 26 byte log entry
0709 12:17:06.488655 (+445661us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:06.488711 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:06.488807 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:06.991102 (+502295us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:07.247160 (+256058us) raft_consensus.cc:1685] finished
0709 12:17:07.247190 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:07.311420 (+ 64230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9637,"spinlock_wait_cycles":6003968,"thread_start_us":404772,"threads_started":1,"wal-append.queue_time_us":537310}
I20260709 12:17:07.263867 20125 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:17:07.275556 20019 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.310720 19982 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.268816 20039 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.313153 20059 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.361097 20152 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.361256 20130 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.363552 19977 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:07.391760 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 150372us
W20260709 12:17:07.394259 20137 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 12:17:07.388196 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 159128us
I20260709 12:17:07.314282 19972 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.336724 20129 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.401515 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:07.334877 20151 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:07.390113 20079 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:07.402572 20142 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:17:07.402658 19989 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.403404 19989 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.403779 20049 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.402287 18374 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:07.405997 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:07.406625 18374 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:07.406672 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:07.407315 19041 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.406977 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:07.409641 18369 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:07.412636 18374 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:07.413264 18443 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:07.324311 20148 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:17:07.424890 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:07.425757 18374 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:07.427435 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:07.428179 18369 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.431778 19041 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:07.409942 20079 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:07.433936 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:07.431790 19978 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:07.439397 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:07.403798 19989 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.434666 20151 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:07.439427 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:07.448246 18368 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:07.454042 20129 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:07.455858 20145 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:07.396627 20152 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:07.464617 20059 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.440402 20049 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:07.409576 20130 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:17:07.469904 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:07.439040 20053 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:07.475406 19041 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:07.464987 20039 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.507205 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1826) took 1187 ms. Trace:
I20260709 12:17:07.507702 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 236) took 1020 ms. Trace:
I20260709 12:17:07.508279 18296 rpcz_store.cc:276] 0709 12:17:06.486857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.487013 (+   156us) service_pool.cc:224] Handling call
0709 12:17:06.490811 (+  3798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:06.590496 (+ 99685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:06.590538 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:06.615038 (+ 24500us) log.cc:844] Serialized 26 byte log entry
0709 12:17:06.952906 (+337868us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:06.952958 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:06.953041 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:07.507435 (+554394us) raft_consensus.cc:1685] finished
0709 12:17:07.507464 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:07.507631 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2126,"raft.queue_time_us":34707,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":24331,"thread_start_us":330185,"threads_started":2,"wal-append.queue_time_us":598941}
I20260709 12:17:07.473426 19993 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.459474 20152 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.440723 20079 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.446182 19978 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.466786 20059 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:07.487033 20049 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:07.507647 18445 rpcz_store.cc:276] 0709 12:17:06.319768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.319979 (+   211us) service_pool.cc:224] Handling call
0709 12:17:06.320296 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:06.725073 (+404777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:06.725115 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:06.725709 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:17:07.225535 (+499826us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:07.225600 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:07.225691 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:07.506971 (+281280us) raft_consensus.cc:1685] finished
0709 12:17:07.506993 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:07.507142 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55769,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":267648,"thread_start_us":816203,"threads_started":2,"wal-append.queue_time_us":659748}
I20260709 12:17:07.445281 19977 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:07.525192 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90008.4us
I20260709 12:17:07.642743 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:07.622958 20147 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:17:07.512468 19972 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.550899 20040 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.634967 19982 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.504540 20053 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.659111 20049 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.635171 20148 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:07.675010 20053 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.675073 20148 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.676592 19982 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:07.635427 20039 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.670475 18368 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:07.551406 20028 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.586508 20019 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.553695 19971 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:07.675851 20053 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:07.638980 20135 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.668051 20130 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.685035 20039 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.677295 19982 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.687812 19989 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.695828 18369 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:07.698179 20091 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:07.712404 19982 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.713261 19983 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:07.713959 19983 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.716805 19983 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.718508 19983 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:07.724368 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 472966us
I20260709 12:17:07.704205 20019 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:07.721222 20091 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.679064 20040 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.706125 19971 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:07.705476 20053 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.593551 20151 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.729791 19989 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:07.735149 20091 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.731946 20148 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:07.724454 19983 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.737934 20019 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:07.811851 20019 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:07.819490 20079 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.805972 19971 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:08.163604 20040 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:08.164279 19978 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.748497 19989 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:08.167126 19989 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:08.172616 19983 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:07.787884 20039 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:08.325338 20091 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.000s
I20260709 12:17:08.326057 20091 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:08.328891 20091 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:08.523217 20053 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:08.885622 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.179s	user 0.000s	sys 0.031s
W20260709 12:17:08.887367 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.000s	sys 0.032s
W20260709 12:17:08.892345 20127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.643s	user 0.008s	sys 0.036s
W20260709 12:17:08.892875 20127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.650s	user 0.008s	sys 0.036s
W20260709 12:17:08.894603 20032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.524s	user 0.011s	sys 0.011s
W20260709 12:17:08.895361 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.017s	sys 0.015s
W20260709 12:17:08.915555 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.019s	sys 0.015s
W20260709 12:17:08.916251 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.18908e+06us
W20260709 12:17:08.895620 20141 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.004s	sys 0.007s
W20260709 12:17:08.895222 20032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.525s	user 0.011s	sys 0.011s
I20260709 12:17:08.932189 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:08.932674 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:08.938314 19043 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:08.895521 20139 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.935806 20063 tablet_copy_client.cc:538] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:17:08.956570 20151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.141s	user 0.005s	sys 0.032s
W20260709 12:17:08.957266 20151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.005s	sys 0.032s
W20260709 12:17:08.935847 19880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.420s	user 0.008s	sys 0.029s
W20260709 12:17:08.958770 19880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.448s	user 0.009s	sys 0.030s
I20260709 12:17:08.944541 20139 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:08.958252 20141 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.505s	user 0.004s	sys 0.008s
W20260709 12:17:08.961380 19008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.006s	sys 0.012s
W20260709 12:17:08.954983 19977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.136s	user 0.010s	sys 0.012s
W20260709 12:17:08.967862 19008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.008s	sys 0.013s
W20260709 12:17:08.960706 20049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.006s	sys 0.015s
W20260709 12:17:08.968086 19977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.011s	sys 0.012s
W20260709 12:17:08.960901 20129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.254s	user 0.006s	sys 0.015s
I20260709 12:17:08.968339 20179 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.962749 20148 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:08.961941 20151 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:08.969017 20091 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:08.973969 19971 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.003s	sys 0.022s
W20260709 12:17:08.972664 20049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.006s	sys 0.015s
I20260709 12:17:08.982497 20139 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.004551 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:09.005957 19971 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.012s	sys 0.029s
I20260709 12:17:09.003189 20148 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.011312 20148 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.004972 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.003273 20151 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.016850 20063 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:17:09.017058 20187 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.018668 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.020602 20187 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.021404 20151 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.023080 20187 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.024524 20187 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.025120 20187 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.023773 20049 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:09.003686 20129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.297s	user 0.006s	sys 0.019s
I20260709 12:17:08.979748 19880 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.027393 20187 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.005954 20179 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.033317 20187 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.032719 19880 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.037619 19880 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.005888 20091 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.040498 20187 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.046697 20187 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.047390 20187 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.048314 20187 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.049515 20188 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.043706 19880 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.051646 20188 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.052289 19880 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.054364 20188 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.030517 20049 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.054803 19880 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.056897 20049 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:09.055176 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.691s	user 0.000s	sys 0.017s
W20260709 12:17:09.058595 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.695s	user 0.000s	sys 0.020s
W20260709 12:17:09.059175 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.69736e+06us
I20260709 12:17:09.064445 20187 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.066325 20092 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.075825 20092 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.077934 20092 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.066601 19990 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.078728 19990 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.081094 19990 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.107971 20032 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.109375 20127 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.109228 20032 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.109926 20127 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.113750 20139 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.114248 20139 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.115360 20127 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.100026 20168 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:09.107098 20131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.799s	user 0.022s	sys 0.023s
I20260709 12:17:09.112975 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.111461 20177 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.115973 20092 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.116106 20168 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.116374 20177 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.116643 20092 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.118225 20168 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.118353 20177 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.106873 19990 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.111670 20049 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.118892 19990 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.118960 20049 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.111454 20187 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.123885 19880 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.159088 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.168159 18444 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:09.163406 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.169584 18446 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.186379 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.195851 20195 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.196576 20195 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.211915 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.229624 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.230223 18443 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:09.113046 20032 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:09.116123 20131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.812s	user 0.022s	sys 0.023s
I20260709 12:17:09.155866 20177 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:09.233433 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.124265 20188 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.130561 20091 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.239053 18446 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
W20260709 12:17:09.187106 18412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.124s (SENT) [suppressed 25 similar messages]
I20260709 12:17:09.143687 20193 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.240543 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:09.243966 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:09.211593 19880 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.245613 20081 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:09.247360 20081 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.231467 20168 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.231904 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.248118 20193 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.247726 20081 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.255079 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:09.258898 19880 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.240878 20195 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.244262 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.262341 20076 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:09.244869 20177 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:09.263604 20177 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.252130 20091 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:09.211194 20187 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.287433 20193 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.288594 18446 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:09.289330 20187 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.290145 20148 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:09.291517 18415 connection.cc:256] RPC call timeout handler was delayed by -1.4023604399999385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 12:17:09.292874 18415 connection.cc:256] RPC call timeout handler was delayed by -1.2212771299998622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
I20260709 12:17:09.244537 20188 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.267690 20091 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.272939 20076 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:09.301218 19990 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.301738 20076 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.304347 20196 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.304873 20196 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.298630 20195 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.312036 19880 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.271541 20168 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.300709 20091 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:09.298497 20131 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.312646 19880 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.304724 20193 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.316154 20195 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.307454 20187 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.329425 19880 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.330469 19880 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.331115 19880 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:09.332957 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 145878us
I20260709 12:17:09.333395 19880 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.324393 20092 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.323186 20197 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.335326 20197 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.336359 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.324559 20131 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:09.337112 18443 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:09.338923 20188 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.339080 20131 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.339545 20187 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.340979 20049 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.341888 20187 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.344480 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.346653 18443 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:09.351745 20195 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.353822 20168 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.325246 20198 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.357333 20196 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.330222 20193 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.365720 20193 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.366153 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.366823 18441 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
W20260709 12:17:09.367373 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.925s	user 0.011s	sys 0.004s
W20260709 12:17:09.367833 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.925s	user 0.011s	sys 0.004s
W20260709 12:17:09.368162 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 1.92598e+06us
I20260709 12:17:09.370997 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.375291 20193 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.385063 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.393450 20197 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.406821 20197 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.406747 20177 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.407671 20168 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.407932 20177 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.408293 20168 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.410277 20177 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:09.410912 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2829656839999188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 12:17:09.407672 20197 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.415436 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.416222 18374 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:09.416277 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.417106 18443 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:09.413574 20197 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.421578 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.422415 18443 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:09.419278 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:09.421034 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.427870 19042 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:09.428323 18368 raft_consensus.cc:2410] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:09.430596 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.431175 18374 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:09.410281 20168 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.415261 19993 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.417879 20196 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.368490 20198 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.386803 20193 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.482620 20081 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.483323 19993 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.496675 19993 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.497311 20091 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.489662 20081 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.481379 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.487432 20196 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.503177 20198 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.504931 20193 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.509797 20032 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.505796 19853 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.506705 19990 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.515682 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:09.516759 19006 connection.cc:256] RPC call timeout handler was delayed by -1.230415751999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 12:17:09.517712 20148 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.517895 19993 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.509935 20168 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.509116 20177 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.510113 20129 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.463539 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.549818 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.552929 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.553500 18374 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:09.556879 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.519204 20148 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.560714 20148 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.560781 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.561161 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.562005 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.562794 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.563364 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.565006 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.563855 20129 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.567656 20207 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:09.568187 20196 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.569695 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.569928 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.569937 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.570319 18446 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:09.571039 18375 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:09.572575 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.572549 18441 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:09.574545 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.551661 20168 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.576661 20198 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:09.577169 20198 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:09.577548 20168 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.578418 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.579574 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.582854 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.583691 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.584589 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:09.589262 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.589341 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.590142 18375 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:09.552155 20177 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.591617 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.591794 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.593010 18375 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:09.593680 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.595145 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.597018 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.597028 20197 raft_consensus.cc:2764] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:09.574167 20193 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:09.577957 20198 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.598304 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.599084 18375 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:09.600215 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.598363 20193 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:09.601236 18441 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:09.601890 20193 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.597391 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:09.604321 20206 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:09.604429 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.604710 20206 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.605121 20206 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.605940 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.607410 20207 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:09.607765 20207 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.608194 20207 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.601343 20177 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.612327 20197 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:09.612709 20197 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.613077 20197 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:09.615254 20193 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.619891 20193 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.623390 20211 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:09.624025 20211 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.624572 20211 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.627550 20214 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:09.628014 20214 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.628549 20214 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:09.638587 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62770.2us
W20260709 12:17:09.639765 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49363.1us
I20260709 12:17:09.642529 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.643563 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.644276 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.644747 20193 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:09.645972 20195 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:09.646144 20193 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.646517 20195 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.646673 20193 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:09.647042 20195 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.645135 20196 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:09.650966 20196 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.651422 20196 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.652616 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.664011 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.664178 20215 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:09.664238 20207 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.664805 20177 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:09.664872 20215 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.665236 20177 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.665398 20215 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.665699 20177 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:09.667491 20207 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.670835 20193 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.673656 20214 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.653859 20197 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.654773 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.676879 19035 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:09.678483 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.679576 19035 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:09.660866 20206 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.658751 20196 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.681612 20195 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.690878 20179 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:09.683672 18264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.481s	user 0.046s	sys 0.055s
W20260709 12:17:09.692023 18264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.490s	user 0.047s	sys 0.055s
I20260709 12:17:09.692358 20197 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.686544 20206 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.688607 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.701279 20196 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:09.689531 20193 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:09.692350 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 2.49196e+06us
I20260709 12:17:09.694427 20195 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:09.695730 18373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.260s	user 0.010s	sys 0.036s
W20260709 12:17:09.696308 18266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.385s	user 0.015s	sys 0.020s
W20260709 12:17:09.705137 18266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.394s	user 0.015s	sys 0.021s
W20260709 12:17:09.705521 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 2.39855e+06us
I20260709 12:17:09.687353 20215 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.701597 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.709609 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:09.712155 19035 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:09.715337 20215 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.716689 20214 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.717326 20215 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.717851 20215 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:09.704345 18373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.269s	user 0.010s	sys 0.036s
I20260709 12:17:09.720323 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1321) took 2299 ms. Trace:
I20260709 12:17:09.720403 20215 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.720670 18373 rpcz_store.cc:276] 0709 12:17:07.422422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:07.422558 (+   136us) service_pool.cc:224] Handling call
0709 12:17:07.431398 (+  8840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:07.431714 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:07.431739 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:07.432751 (+  1012us) log.cc:844] Serialized 26 byte log entry
0709 12:17:09.719990 (+2287239us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:09.720027 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:09.720106 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:09.720145 (+    39us) raft_consensus.cc:1685] finished
0709 12:17:09.720155 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:09.720253 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23051,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":2284472,"threads_started":1,"wal-append.queue_time_us":1450767}
I20260709 12:17:09.722646 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.723409 20218 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:09.728557 20222 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.729486 20221 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.729308 20222 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.730075 20221 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.732362 20221 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.732851 20221 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.733311 20221 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.698539 20177 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.665845 20211 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.679867 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:09.738770 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58977.4us
I20260709 12:17:09.740136 20217 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:09.740545 20217 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.740931 20217 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:09.740878 20224 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.741927 20224 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.744530 20224 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.745404 20224 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.745858 20224 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.747702 20224 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.735343 20221 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.739470 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.755961 20226 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.757174 20221 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.788244 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.777535 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.788605 20221 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.789351 18296 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:09.790325 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:09.778834 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 43798.8us
I20260709 12:17:09.765672 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.792403 18299 raft_consensus.cc:2410] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:09.758900 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.788981 20217 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.793527 18292 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:09.749392 20224 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:09.783490 20121 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:09.800756 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.801263 20121 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:09.801393 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.802446 20121 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.802781 18292 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
W20260709 12:17:09.803859 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53539.1us
I20260709 12:17:09.808601 20228 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.809360 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.809300 20228 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.810345 18442 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:09.812165 20228 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.812855 20228 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.813406 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.814088 18442 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.805045 20224 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.813539 20177 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:09.859717 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67333.9us
I20260709 12:17:09.805577 20222 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.799784 20226 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.815775 20214 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:17:09.886582 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.887362 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.891119 18293 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:09.892153 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.892159 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.893236 18441 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 3 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:09.894676 18442 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:09.895282 18446 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:09.900413 20224 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.887562 20222 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.915079 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:09.934271 20231 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.935017 20231 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.937341 20226 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.886046 20228 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.897562 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:09.937915 20226 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.939113 18293 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:09.941449 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:09.942202 18293 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.944963 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.895810 20221 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.945827 18442 raft_consensus.cc:2368] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:09.947089 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:09.948232 18299 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:09.949596 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:09.947521 20214 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:09.950577 18441 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:09.950817 20214 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:09.951858 20231 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.951781 20214 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.952099 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.953578 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:09.954208 18366 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:09.938913 20217 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.938308 20230 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:09.956595 20230 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.958043 18194 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 293) took 4185 ms. Trace:
I20260709 12:17:09.958704 20230 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.959897 18446 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:09.966853 18339 leader_election.cc:400] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:09.967298 18339 leader_election.cc:403] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:09.958403 18194 rpcz_store.cc:276] 0709 12:17:05.772280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.772435 (+   155us) service_pool.cc:224] Handling call
0709 12:17:09.957960 (+4185525us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:06.416232 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:06.417008 (+   776us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:06.417048 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:17:06.417068 (+    20us) tablet.cc:662] Decoding operations
0709 12:17:06.417409 (+   341us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:06.417470 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:06.417498 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:06.417669 (+   171us) tablet.cc:701] Row locks acquired
0709 12:17:06.417687 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:17:06.417844 (+   157us) write_op.cc:270] Start()
0709 12:17:06.417950 (+   106us) write_op.cc:276] Timestamp: P: 1783599426417799 usec, L: 0
0709 12:17:06.417971 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:17:06.559220 (+141249us) log.cc:844] Serialized 541 byte log entry
0709 12:17:06.744716 (+185496us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e77e28 00007f3c6c8b2581 00007f3c6c8b1d58 00007f3c6c8b69f1 00007f3c6c8b69a9 00007f3c6c8b6939 00007f3c6c8b6901 00007f3c6c8b5b2d 0000000000388634 0000000000388479
0709 12:17:07.431150 (+686434us) op_driver.cc:464] REPLICATION: finished
0709 12:17:08.973127 (+1541977us) write_op.cc:301] APPLY: starting
0709 12:17:08.975289 (+  2162us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:08.984445 (+  9156us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:08.984973 (+   528us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:09.001114 (+ 16141us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:09.006476 (+  5362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:09.006541 (+    65us) write_op.cc:312] APPLY: finished
0709 12:17:09.007693 (+  1152us) log.cc:844] Serialized 23 byte log entry
0709 12:17:09.035586 (+ 27893us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:09.035734 (+   148us) write_op.cc:454] Released schema lock
0709 12:17:09.035796 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:09.036263 (+   467us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:09.036305 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":78585,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":1531656,"apply.run_cpu_time_us":16975,"apply.run_wall_time_us":65707,"num_ops":1,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":11729,"prepare.run_wall_time_us":328661,"replication_time_us":1003684,"spinlock_wait_cycles":148916992,"thread_start_us":1554887,"threads_started":2,"wal-append.queue_time_us":36030}]]}
I20260709 12:17:09.969857 20229 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:09.967196 20121 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.970328 20228 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:09.975605 20177 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:09.985077 20225 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:09.940140 20222 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:09.943553 20226 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.990077 20225 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.002513 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1802) took 3001 ms. Trace:
I20260709 12:17:10.002996 18300 rpcz_store.cc:276] 0709 12:17:07.000812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:07.001198 (+   386us) service_pool.cc:224] Handling call
0709 12:17:07.001508 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:07.012016 (+ 10508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:07.012052 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:07.022665 (+ 10613us) log.cc:844] Serialized 26 byte log entry
0709 12:17:07.721823 (+699158us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:07.721911 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:07.721995 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:10.002197 (+2280202us) raft_consensus.cc:1685] finished
0709 12:17:10.002226 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.002410 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10128,"raft.queue_time_us":210328,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":178,"thread_start_us":453153,"threads_started":1,"wal-append.queue_time_us":1653799}
I20260709 12:17:10.008080 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.017377 18293 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:10.017570 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.019500 18300 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:09.528308 19993 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:09.522713 20091 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.038926 18366 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.049587 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.051612 18297 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:10.053416 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.057801 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.059657 18297 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:10.063378 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.067240 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.074436 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.078006 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.080006 18297 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:10.082038 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.085300 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.087565 19993 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.088035 20121 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.089421 20226 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.090824 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.091583 18366 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.094063 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.086838 18297 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:10.101788 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.103523 18300 raft_consensus.cc:2376] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3. Re-sending same reply.
I20260709 12:17:10.107298 18366 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.109778 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.110328 18300 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.111449 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.097317 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.113790 18293 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:10.114593 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.115131 18366 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.115749 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:10.120237 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.120869 18297 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:10.122148 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.122640 20222 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.126452 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.128114 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:10.129886 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:10.129801 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.123423 20187 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:10.131398 18293 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:10.129087 18366 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:10.132133 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.133090 20227 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:10.133885 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.135839 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:09.514858 20032 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.136457 18366 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:10.137562 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.138698 18373 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:10.140625 19993 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:10.142513 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146705us
I20260709 12:17:10.143743 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.145220 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.145887 18442 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:10.147060 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.147645 18446 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.147830 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.148375 18297 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.149086 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.149574 18293 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:10.143220 20187 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:10.151283 20187 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.152474 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.153091 18449 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:10.139361 20121 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:10.154220 20121 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.154481 20121 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.133834 20227 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:10.138170 20032 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.128181 20234 raft_consensus.cc:2764] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:10.163935 18446 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.160805 20227 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.164534 20227 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.166240 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.166760 18297 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:10.169811 18300 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:10.172034 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.173571 20229 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:10.175419 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:10.176472 19993 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:10.176930 19993 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.177347 19993 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.178930 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.180302 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.180621 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.181375 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.184110 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.184849 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.188601 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.190968 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.188509 20234 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:10.192099 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.192108 20214 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:10.196430 20214 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:10.191697 20032 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.196841 20214 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:10.197098 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.193819 20238 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:17:10.197925 20168 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:10.198423 20168 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:10.199355 20168 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.200495 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.201686 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.185042 20229 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:10.207017 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.207274 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:10.211360 20200 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.840s	user 0.005s	sys 0.007s
I20260709 12:17:10.211347 18293 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:10.213510 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.214577 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.215337 20236 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:10.215606 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.215755 20236 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:10.216153 20236 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.216324 18367 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:10.219130 18366 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:10.225437 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 350) took 1039 ms. Trace:
I20260709 12:17:10.225826 18444 rpcz_store.cc:276] 0709 12:17:09.185781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.185927 (+   146us) service_pool.cc:224] Handling call
0709 12:17:09.186269 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:09.251703 (+ 65434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:09.251745 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:09.257650 (+  5905us) log.cc:844] Serialized 26 byte log entry
0709 12:17:09.313204 (+ 55554us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:09.313240 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:09.313306 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:10.187958 (+874652us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b24001e9c38. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dce09d 00007f3c68a0010e 00007f3c68a1788c 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221
0709 12:17:10.217420 (+ 29462us) raft_consensus.cc:1685] finished
0709 12:17:10.217434 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.225373 (+  7939us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45854,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":68,"spinlock_wait_cycles":38346496,"thread_start_us":93257,"threads_started":2,"wal-append.queue_time_us":77690}
I20260709 12:17:10.217048 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.227470 18367 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:10.229796 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.230700 18367 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:10.231482 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.232035 18367 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:10.232688 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.233469 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.229789 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:10.209288 18368 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.236332 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.236609 20076 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:10.236982 20076 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.237316 20076 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.242853 20214 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.244742 18367 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:10.254694 20214 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.263922 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.264549 18444 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:10.234015 20121 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.266718 20121 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.267478 18371 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:10.267774 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.269241 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:10.272460 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.757s	user 0.019s	sys 0.021s
I20260709 12:17:10.271945 18366 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:10.272964 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.758s	user 0.019s	sys 0.022s
I20260709 12:17:10.276052 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.278831 18444 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:10.268720 18293 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:10.280100 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.284694 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.285804 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.284799 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.288085 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.288403 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.288784 18293 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.289384 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.284578 20121 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:10.291953 20121 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:10.290163 19043 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:17:10.261710 20227 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.292894 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.294298 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 102269us
I20260709 12:17:10.295984 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.296168 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.299345 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.313778 18291 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.314006 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.299441 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.300794 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.309013 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.287374 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.299015 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.315711 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.294067 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.311692 19993 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.324575 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.326787 18263 connection.cc:256] RPC call timeout handler was delayed by -1.4419928500001333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 12:17:10.324432 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.332798 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.333990 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.334587 18299 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:10.330451 20239 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.337823 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.339375 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.339409 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.338433 18299 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:10.341418 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.345464 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.348012 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:10.348763 19041 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.357754 19041 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.358896 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.384078 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.359927 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.360898 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.407590 18299 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:10.363834 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.365566 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.330276 20240 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.330514 20226 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:10.367365 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:10.441391 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.441861 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 74614.7us
I20260709 12:17:10.442430 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.371783 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75858us
I20260709 12:17:10.382009 18293 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.436394 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.437932 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.339180 20214 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:10.445436 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.458494 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 119202us
I20260709 12:17:10.437140 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.466421 20243 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:10.466759 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.468479 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.465381 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.468816 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.445510 18299 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:10.469738 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.472522 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:10.446326 20230 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:10.497982 20230 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:10.498106 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.465510 20237 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.476620 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.498769 18299 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.499320 18369 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:10.498544 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.499745 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1326) took 1084 ms. Trace:
I20260709 12:17:10.500116 18369 rpcz_store.cc:276] 0709 12:17:09.415313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.468235 (+1052922us) service_pool.cc:224] Handling call
0709 12:17:10.499692 (+ 31457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.500869 19041 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:10.501060 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.502332 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.498497 20230 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:10.501260 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.503779 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.505455 18299 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:10.503702 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.505515 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.475551 20227 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.506317 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.504755 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:10.508229 18369 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:10.370035 20121 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.515309 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:10.516127 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.516265 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.525513 20249 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:10.444200 20236 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.458302 20226 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.547346 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.548588 19040 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:10.540000 20227 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:10.550835 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45819.1us
I20260709 12:17:10.538311 20223 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:10.537807 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:10.560031 18369 raft_consensus.cc:606] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:10.628979 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.629829 18297 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.552951 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:10.640101 20076 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.637087 20237 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.632264 19993 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.636847 20249 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:10.638824 20226 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:10.599404 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08000cda00 after lost signal to thread 17160
I20260709 12:17:10.639853 20236 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.642262 20076 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.646556 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:10.646561 20245 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:17:10.647681 18369 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:10.632756 20223 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:10.653172 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.657207 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.659112 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 240) took 1150 ms. Trace:
I20260709 12:17:10.659169 18444 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.659439 18295 raft_consensus.cc:2368] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:10.659509 18369 rpcz_store.cc:276] 0709 12:17:09.508569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.646978 (+1138409us) service_pool.cc:224] Handling call
0709 12:17:10.647513 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.658862 (+ 11349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.659055 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.661254 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.662535 18449 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.664590 18369 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:10.665261 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1339) took 1137 ms. Trace:
I20260709 12:17:10.658576 20243 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:10.665614 18369 rpcz_store.cc:276] 0709 12:17:09.528117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.660371 (+1132254us) service_pool.cc:224] Handling call
0709 12:17:10.660667 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.665040 (+  4373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.665214 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.667521 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.666900 20243 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.666909 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 244) took 1138 ms. Trace:
I20260709 12:17:10.668594 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.669203 18295 raft_consensus.cc:2376] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4. Re-sending same reply.
I20260709 12:17:10.668537 18369 rpcz_store.cc:276] 0709 12:17:09.528778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.666457 (+1137679us) service_pool.cc:224] Handling call
0709 12:17:10.666638 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.666739 (+   101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.666752 (+    13us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.666777 (+    25us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.666793 (+    16us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.666821 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.666877 (+    56us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.672052 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 247) took 1115 ms. Trace:
I20260709 12:17:10.672420 18369 rpcz_store.cc:276] 0709 12:17:09.556453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.671308 (+1114855us) service_pool.cc:224] Handling call
0709 12:17:10.671601 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.671756 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.671780 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.671826 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.671867 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.671916 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.672006 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.675789 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.676471 18295 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:10.677472 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.677157 18449 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.678244 18295 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.664815 20228 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:10.683504 20258 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:10.684634 20252 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:17:10.691625 18366 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:10.696648 18444 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 3 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:10.687713 20228 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.700501 20223 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.701370 20228 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.692167 20249 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.712293 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:10.726708 18448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 1838) took 3774 ms (client timeout 2879 ms). Trace:
W20260709 12:17:10.768759 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92751.3us
W20260709 12:17:10.727411 18448 rpcz_store.cc:269] 0709 12:17:06.933671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.933773 (+   102us) service_pool.cc:224] Handling call
0709 12:17:10.708279 (+3774506us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54001856d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:10.708472 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715520896}
I20260709 12:17:10.767515 20262 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:10.798799 19037 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:10.800535 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.804345 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:10.805154 19044 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.808966 20262 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.809608 20262 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.832732 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:10.839751 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1501) took 1116 ms. Trace:
I20260709 12:17:10.840291 18371 rpcz_store.cc:276] 0709 12:17:09.723609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.201415 (+477806us) service_pool.cc:224] Handling call
0709 12:17:10.839495 (+638080us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b5400413f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:10.839653 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149782272}
I20260709 12:17:10.842235 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 251) took 1144 ms. Trace:
I20260709 12:17:10.850574 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.842638 18371 rpcz_store.cc:276] 0709 12:17:09.697575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.841231 (+1143656us) service_pool.cc:224] Handling call
0709 12:17:10.841594 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.841814 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.841869 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.841927 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.841970 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.842049 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.842173 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:10.864840 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 165784us
I20260709 12:17:10.867331 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:10.867466 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.869055 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:10.869166 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1352) took 1223 ms. Trace:
I20260709 12:17:10.870689 18296 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:10.871223 18296 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:10.871495 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 195254us
I20260709 12:17:10.872529 18296 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.873530 18296 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.874245 20225 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:09.528092 19990 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.876648 19990 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.877246 18374 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:10.877063 20230 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.879680 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.879319 18446 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:10.869701 18369 rpcz_store.cc:276] 0709 12:17:09.645435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.674418 (+1028983us) service_pool.cc:224] Handling call
0709 12:17:10.674730 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:10.681723 (+  6993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.681753 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:10.685409 (+  3656us) log.cc:844] Serialized 52 byte log entry
0709 12:17:10.759897 (+ 74488us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:10.868635 (+108738us) log.cc:844] Serialized 14 byte log entry
0709 12:17:10.868858 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.868924 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:10.868971 (+    47us) raft_consensus.cc:1685] finished
0709 12:17:10.868982 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.869097 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8034,"prepare.queue_time_us":105951,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":216,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3312,"spinlock_wait_cycles":316288,"thread_start_us":115481,"threads_started":2,"wal-append.queue_time_us":100205}
I20260709 12:17:10.882922 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:10.884418 18371 raft_consensus.cc:606] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:10.874169 20269 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.890841 18369 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:10.891692 20268 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.892803 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 252) took 1195 ms. Trace:
I20260709 12:17:10.893844 18369 rpcz_store.cc:276] 0709 12:17:09.697816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.890451 (+1192635us) service_pool.cc:224] Handling call
0709 12:17:10.890696 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.892597 (+  1901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.892756 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.893164 18296 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:10.894968 18369 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:10.897621 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 253) took 1199 ms. Trace:
I20260709 12:17:10.897955 18369 rpcz_store.cc:276] 0709 12:17:09.698033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.894460 (+1196427us) service_pool.cc:224] Handling call
0709 12:17:10.894728 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.895291 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.897580 (+  2289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.893903 20230 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.897249 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.899294 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 255) took 1200 ms. Trace:
I20260709 12:17:10.899765 20268 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.900215 18295 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.899680 18369 rpcz_store.cc:276] 0709 12:17:09.698521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.898586 (+1200065us) service_pool.cc:224] Handling call
0709 12:17:10.898815 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.898963 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.898984 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.899021 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.899052 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.899101 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.899191 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.901710 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.894572 20269 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.903904 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.892107 20271 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.892277 20258 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:10.907105 18371 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:10.908685 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1505) took 1183 ms. Trace:
I20260709 12:17:10.908931 18371 rpcz_store.cc:276] 0709 12:17:09.725710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.903792 (+1178082us) service_pool.cc:224] Handling call
0709 12:17:10.906905 (+  3113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.908514 (+  1609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.908648 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.906721 20271 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.912137 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:10.915519 20273 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:17:10.913630 18294 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:10.918771 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1818) took 1150 ms. Trace:
I20260709 12:17:10.918915 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.919385 18294 rpcz_store.cc:276] 0709 12:17:09.768614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.911908 (+1143294us) service_pool.cc:224] Handling call
0709 12:17:10.918715 (+  6807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.917068 20228 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.929387 20211 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:10.931507 20228 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.932916 20269 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.918746 20268 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.933980 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.935730 19044 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:10.939543 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.939836 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.940922 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.942344 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:10.943811 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.946298 18198 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 209) took 5089 ms. Trace:
W20260709 12:17:10.947675 19044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 627) took 1515 ms (client timeout 1804 ms). Trace:
I20260709 12:17:10.949682 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:10.950394 19040 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:10.950875 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1858) took 1518 ms. Trace:
I20260709 12:17:10.951395 18443 rpcz_store.cc:276] 0709 12:17:09.432171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.451392 (+ 19221us) service_pool.cc:224] Handling call
0709 12:17:09.574807 (+123415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:09.575086 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:09.589907 (+ 14821us) log.cc:844] Serialized 14 byte log entry
0709 12:17:10.923936 (+1334029us) spinlock_profiling.cc:243] Waited 971 us on lock 0x7b1800259900. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:17:10.950362 (+ 26426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.950434 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:10.950489 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.950655 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.950804 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2168,"prepare.queue_time_us":13511,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":121,"spinlock_wait_cycles":2234240,"thread_start_us":23977,"threads_started":1}
I20260709 12:17:10.953675 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.954350 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:10.954392 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:10.949275 19044 rpcz_store.cc:269] 0709 12:17:09.420556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.420687 (+   131us) service_pool.cc:224] Handling call
0709 12:17:10.804009 (+1383322us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:10.936275 (+132266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6633728}
I20260709 12:17:10.957329 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.958428 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:10.958729 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.945631 20081 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:10.938668 20160 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:10.951422 20269 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.946902 18198 rpcz_store.cc:276] 0709 12:17:05.846841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.857061 (+ 10220us) service_pool.cc:224] Handling call
0709 12:17:06.004095 (+147034us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:17:10.935824 (+4931729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:06.745084 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:06.745602 (+   518us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:06.745634 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:17:06.745649 (+    15us) tablet.cc:662] Decoding operations
0709 12:17:06.746274 (+   625us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:06.746318 (+    44us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:06.746350 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:17:06.746862 (+   512us) tablet.cc:701] Row locks acquired
0709 12:17:06.746880 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:17:06.747016 (+   136us) write_op.cc:270] Start()
0709 12:17:06.747134 (+   118us) write_op.cc:276] Timestamp: P: 1783599426746970 usec, L: 0
0709 12:17:06.747158 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:17:07.403476 (+656318us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b5400030820. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df9520 00007f3c68e82b42 00007f3c68e77f67 00007f3c68e77da4 00007f3c6c8b2581 00007f3c6c8b1d58 00007f3c6c8b69f1 00007f3c6c8b69a9 00007f3c6c8b6939 00007f3c6c8b6901
0709 12:17:07.404196 (+   720us) log.cc:844] Serialized 2069 byte log entry
0709 12:17:09.360631 (+1956435us) op_driver.cc:464] REPLICATION: finished
0709 12:17:09.609749 (+249118us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54003ebda8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6f2389e1 00007f3c6f23764c 00007f3c6c8b3845 00007f3c6c8b7d61 00007f3c6c8b7d19 00007f3c6c8b7ca9 00007f3c6c8b7c71 00007f3c6c8b6e9d 0000000000388634 0000000000388479 00007f3c65d8f475
0709 12:17:09.609820 (+    71us) write_op.cc:301] APPLY: starting
0709 12:17:09.609922 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:09.610102 (+   180us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:09.610123 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:09.610808 (+   685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:09.610981 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:17:09.611023 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:09.611260 (+   237us) log.cc:844] Serialized 47 byte log entry
0709 12:17:09.612562 (+  1302us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:09.612665 (+   103us) write_op.cc:454] Released schema lock
0709 12:17:09.612706 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:09.613037 (+   331us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:09.613069 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":374754,"reported_tablets":5,"spinlock_wait_cycles":99581312,"child_traces":[["op",{"apply.queue_time_us":4538,"apply.run_cpu_time_us":4395,"apply.run_wall_time_us":220737,"num_ops":5,"prepare.queue_time_us":252317,"prepare.run_cpu_time_us":15689,"prepare.run_wall_time_us":663299,"replication_time_us":2609162,"spinlock_wait_cycles":44717696}]]}
I20260709 12:17:10.962373 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.959640 20228 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.967450 20273 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:10.968063 20276 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:10.968766 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.972822 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.967526 20081 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:10.970503 20091 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.974094 20081 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:10.974331 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 346) took 1813 ms. Trace:
I20260709 12:17:10.975672 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.925024 20258 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:10.974727 18447 rpcz_store.cc:276] 0709 12:17:09.160672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.973571 (+1812899us) service_pool.cc:224] Handling call
0709 12:17:10.973881 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.974054 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.974077 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.974116 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.974144 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.974194 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.974282 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:10.963111 20271 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:10.982364 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:10.982887 19040 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:10.978942 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 249) took 1416 ms. Trace:
I20260709 12:17:10.983642 19039 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:17:10.970439 20076 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:10.969386 20277 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:10.989355 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.988780 20280 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:10.969091 20276 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:10.991286 20280 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:10.991456 20276 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.975453 20081 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:10.994026 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.983865 18374 rpcz_store.cc:276] 0709 12:17:09.566218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.594844 (+ 28626us) service_pool.cc:224] Handling call
0709 12:17:10.978774 (+1383930us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540009f6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:10.978871 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2946054400}
I20260709 12:17:10.992964 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:10.995715 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.996788 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:10.997090 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.992030 20280 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:10.968953 20273 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:10.998878 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:10.999254 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.998277 20280 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:10.972129 20160 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.001097 20280 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:10.967587 20269 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.003146 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:10.988215 20091 raft_consensus.cc:2764] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:11.010191 18374 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:11.010309 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.011452 18374 raft_consensus.cc:606] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Received request to transfer leadership to TS 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.012838 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.001715 20218 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.017539 18443 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:11.000563 20228 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.019961 18374 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.026908 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 259) took 1298 ms. Trace:
I20260709 12:17:11.027369 18374 rpcz_store.cc:276] 0709 12:17:09.728621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.019498 (+1290877us) service_pool.cc:224] Handling call
0709 12:17:11.019801 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.026618 (+  6817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.026836 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:11.029273 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 260) took 1041 ms. Trace:
I20260709 12:17:10.998658 20277 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.032033 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.029680 18374 rpcz_store.cc:276] 0709 12:17:09.987860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.028285 (+1040425us) service_pool.cc:224] Handling call
0709 12:17:11.028667 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.028877 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:11.028914 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:11.028970 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:11.029013 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.029084 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:11.029209 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:11.034549 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.034996 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.035189 18442 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:11.036964 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.020514 20230 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:11.038872 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 80240.6us
I20260709 12:17:11.040458 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.016959 20076 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:17:11.045185 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.033198 20277 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.044757 20228 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.038275 20225 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.031524 20262 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.047459 18442 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:17:11.047374 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 93041.2us
I20260709 12:17:11.043404 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.051486 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.046757 20228 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:11.049701 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.051939 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.052863 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:11.045820 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 93893.5us
I20260709 12:17:11.055872 19880 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:11.056238 19880 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.056527 19880 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.050706 20286 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.056416 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:11.057492 20269 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.058212 20285 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.058485 20269 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.059384 20269 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.054710 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.055423 20282 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:11.060581 20269 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.062753 20269 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.063607 19853 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.055855 20228 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.065696 18375 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:11.057715 20225 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.062006 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.066586 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:11.067641 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.070365 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:11.061321 20286 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.071297 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.069614 20078 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.071499 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.071861 20286 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.073525 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.068298 20282 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.075551 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.077565 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.078332 18442 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:11.082516 20262 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.082454 20081 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.083194 20262 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.083101 20285 raft_consensus.cc:2764] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:11.086194 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.086225 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.073774 20078 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:11.087888 20283 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.088155 20218 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.088692 19853 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:11.075838 20225 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:11.090337 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.076333 20282 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:11.074824 20284 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.093719 20283 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.073926 20160 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:11.095526 18442 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 3 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:11.096246 19880 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.075649 20280 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.126076 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:11.133285 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48740.2us
W20260709 12:17:11.148798 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 77364.4us
I20260709 12:17:11.149540 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.150588 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:11.155889 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.155872 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.090488 20262 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.093557 20078 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.158406 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84350.1us
I20260709 12:17:11.158353 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.160166 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.160267 19044 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:11.161020 20032 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:11.161423 20032 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.162178 19037 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.162351 20032 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.163923 19037 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.161358 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.164052 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.140072 20228 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.098477 20283 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:11.165688 18367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1342) took 1609 ms (client timeout 2021 ms). Trace:
I20260709 12:17:11.166180 20076 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.093712 20218 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:11.166929 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.168401 19880 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.121438 20129 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.176136 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.170207 18442 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:11.170279 18372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1335) took 1737 ms (client timeout 1560 ms). Trace:
I20260709 12:17:11.174836 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:11.172058 18367 rpcz_store.cc:269] 0709 12:17:09.556289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.232483 (+676194us) service_pool.cc:224] Handling call
0709 12:17:11.165491 (+933008us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b54000c9ed4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.165623 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26386688}
I20260709 12:17:11.175207 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:11.178496 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:11.178998 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.117333 20284 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.119652 20081 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.179493 18299 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.177767 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.180398 18367 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:11.176918 18372 rpcz_store.cc:269] 0709 12:17:09.432378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.169981 (+1737603us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:11.170089 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:17:11.182052 18298 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.171481 20262 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.186945 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 625) took 1772 ms. Trace:
I20260709 12:17:11.184408 20153 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.187691 20153 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:11.187935 20228 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.188230 20153 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.175482 20218 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.176331 20032 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.193701 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.193787 18372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d4d1763576e4c49849611360f37665a"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:11.194928 18372 raft_consensus.cc:606] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.181641 20230 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.187636 20262 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.180502 20284 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.187206 20276 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:11.191540 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.201102 20076 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:17:11.187364 19036 rpcz_store.cc:276] 0709 12:17:09.414393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.415423 (+  1030us) service_pool.cc:224] Handling call
0709 12:17:11.163953 (+1748530us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57eee 000000000036eb39 00007f3c6ddcb1e0 00007f3c6de940c4 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:11.186882 (+ 22929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3985784576}
I20260709 12:17:11.208912 18299 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:11.181756 20151 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:17:11.213275 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:11.204267 20076 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.219126 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.202356 20230 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.222510 18367 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:11.204330 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.209612 20081 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:17:11.217955 20076 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.229789 20160 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:11.240945 18442 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:11.228876 19037 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:11.243660 18299 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:11.245604 20151 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:11.245996 20151 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.246593 20276 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.229328 20269 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.246798 20151 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.258491 20076 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:11.261771 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 117233us
I20260709 12:17:11.263403 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.265966 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:11.268705 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:11.269631 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 339) took 2113 ms (client timeout 2355 ms). Trace:
W20260709 12:17:11.270051 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.271725 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:11.270066 18440 rpcz_store.cc:269] 0709 12:17:09.156780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.268391 (+2111611us) service_pool.cc:224] Handling call
0709 12:17:11.269563 (+  1172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:11.261613 20280 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.271310 20049 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:11.259336 20282 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.255874 20160 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.268074 20276 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.278652 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.279074 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.275615 20291 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:17:11.280099 20287 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.282758 20081 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:11.283180 20081 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.281981 20049 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:11.281664 18442 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.283078 20160 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.284950 18442 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.284536 20049 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.287729 20264 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.288458 20265 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.289505 20264 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.289853 20264 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.290114 20282 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.290614 20264 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.292013 20264 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.292567 20218 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.292934 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.294581 20218 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.295935 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.296667 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:11.266209 20277 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.299036 20277 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.299785 18447 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:11.271457 20076 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.304831 20265 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.310350 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.313915 20194 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.315955 20269 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.316395 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.319600 20264 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:11.321712 18470 debug-util.cc:398] Leaking SignalData structure 0x7b080009e9e0 after lost signal to thread 18232
I20260709 12:17:11.321863 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:11.322366 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08001184e0 after lost signal to thread 18246
I20260709 12:17:11.324869 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.326871 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.276217 20151 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.330086 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.332152 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:11.334630 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 184097us
I20260709 12:17:11.335520 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.336375 20277 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.336361 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.337667 20078 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.338338 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.338734 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.339452 19040 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.338971 19036 raft_consensus.cc:2376] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3. Re-sending same reply.
I20260709 12:17:11.340821 20277 raft_consensus.cc:2764] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:11.342027 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:11.342898 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.343525 18299 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:11.267731 20230 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.345086 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.347466 19037 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:11.347946 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.348616 20277 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.350787 20139 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:11.351280 20063 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:17:11.340826 20078 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:11.358062 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.337198 20284 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.361426 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.362787 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.365114 18266 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:11.365404 18266 leader_election.cc:403] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:11.366115 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.367606 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.370098 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.345593 20286 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.380699 20218 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:11.373121 20063 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:11.374909 20282 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.369808 20158 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.378257 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.364537 20222 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.372884 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.369295 20017 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.388398 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.381728 20218 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.373585 20225 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:11.391062 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.393606 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.393813 20286 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.394411 20286 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.390386 20218 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.394740 20286 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:11.372223 20078 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:17:11.396238 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.397610 20299 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:17:11.397414 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.388949 20277 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:11.406051 18265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.878s	user 0.022s	sys 0.009s
I20260709 12:17:11.402740 20283 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.404649 20284 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.405016 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.378281 20153 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.401407 20063 tablet_bootstrap.cc:492] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:17:11.406874 18367 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:11.412004 18265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.884s	user 0.023s	sys 0.009s
I20260709 12:17:11.413094 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.414290 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.414597 18368 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:11.415630 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.415937 20206 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.416826 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.396553 20160 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:17:11.418951 20225 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.419106 20160 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.419490 20160 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.419497 20063 ts_tablet_manager.cc:1403] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 2.403s	user 0.021s	sys 0.016s
I20260709 12:17:11.420588 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.417958 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.421115 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 245) took 1869 ms. Trace:
I20260709 12:17:11.422097 18374 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:11.424842 20230 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:11.421581 18366 rpcz_store.cc:276] 0709 12:17:09.551766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.504528 (+952762us) service_pool.cc:224] Handling call
0709 12:17:11.420944 (+916416us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.421061 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337883904}
I20260709 12:17:11.430065 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.442236 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.443394 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.443763 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.445020 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.445856 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.447552 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.448477 18367 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:11.419653 20225 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.451120 20230 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.457239 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.451442 20283 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.451738 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.452010 20077 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.425470 20063 raft_consensus.cc:359] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.414291 20277 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.452813 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.453055 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.466985 20063 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:17:11.450001 20151 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.409132 20218 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.446704 20228 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.406064 20286 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.440111 20160 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.475910 20153 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.477849 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.478515 20160 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.480734 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.467813 20063 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:17:11.472025 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.482913 18366 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.484398 18366 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.485791 18366 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.481586 20284 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:11.488289 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.488588 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.489637 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:11.487064 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:11.491093 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.492605 20302 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.493110 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.495709 20215 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:11.494613 18370 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1334) took 2062 ms (client timeout 1565 ms). Trace:
I20260709 12:17:11.501102 20215 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.503984 20287 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.488314 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.504048 20215 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:11.501263 18370 rpcz_store.cc:269] 0709 12:17:09.432124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.494421 (+2062297us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:11.494552 (+   131us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:17:11.510530 18367 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:11.511864 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.515333 19035 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:11.516924 20148 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.516029 20287 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.517637 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 261) took 1444 ms. Trace:
I20260709 12:17:11.519956 18374 rpcz_store.cc:276] 0709 12:17:10.073752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.033210 (+959458us) service_pool.cc:224] Handling call
0709 12:17:11.421793 (+388583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.422530 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.517462 (+ 94932us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54000f8550. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.517585 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892862848}
I20260709 12:17:11.500397 20063 ts_tablet_manager.cc:1434] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 12:17:11.543879 18446 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.548529 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1876) took 1603 ms. Trace:
I20260709 12:17:11.526139 20284 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:17:11.549448 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.553992 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.556447 19038 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:17:11.560887 19038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 409) took 2365 ms (client timeout 1828 ms). Trace:
I20260709 12:17:11.509178 20302 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.553515 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ca383fa1243547c3b32dfbd6857a3fba received from {username='slave'} at 127.0.0.1:46956
W20260709 12:17:11.561329 19038 rpcz_store.cc:269] 0709 12:17:09.192438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.193268 (+   830us) service_pool.cc:224] Handling call
0709 12:17:11.553834 (+2360566us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:11.556799 (+  2965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9680128}
I20260709 12:17:11.550551 20280 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:11.564092 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ca383fa1243547c3b32dfbd6857a3fba on tablet ca383fa1243547c3b32dfbd6857a3fba with peer 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:11.565639 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119816us
I20260709 12:17:11.549077 18445 rpcz_store.cc:276] 0709 12:17:09.944854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.945175 (+   321us) service_pool.cc:224] Handling call
0709 12:17:09.945558 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:09.945962 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:09.946002 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:11.497351 (+1551349us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:11.497850 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:17:11.546185 (+ 48335us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:11.546229 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.546295 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:11.548221 (+  1926us) raft_consensus.cc:1685] finished
0709 12:17:11.548240 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:11.548466 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":458,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":592,"spinlock_wait_cycles":3175485312,"thread_start_us":47958,"threads_started":1,"wal-append.queue_time_us":48284}
W20260709 12:17:11.568243 18375 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1341) took 2012 ms (client timeout 1991 ms). Trace:
W20260709 12:17:11.568694 18375 rpcz_store.cc:269] 0709 12:17:09.556006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.600003 (+ 43997us) service_pool.cc:224] Handling call
0709 12:17:11.568038 (+1968035us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002f8354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.568175 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3366579072}
I20260709 12:17:11.569059 20302 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.569507 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.570601 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.571434 20153 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.572005 20280 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.572561 20280 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.572093 20153 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.574147 20228 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:11.574501 20222 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:11.574919 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1500) took 1851 ms. Trace:
I20260709 12:17:11.575384 18369 rpcz_store.cc:276] 0709 12:17:09.723316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.901420 (+1178104us) service_pool.cc:224] Handling call
0709 12:17:11.573927 (+672507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.574100 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:11.574123 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:11.574171 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:11.574204 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.574256 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:11.574760 (+   504us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54002f8350. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.574870 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546125184}
I20260709 12:17:11.576998 20225 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.577386 20225 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.577235 20153 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:11.578238 20225 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.579404 20225 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.589116 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.563546 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.590058 18375 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.591485 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.591888 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.593380 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.594041 18298 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:11.595068 19041 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:11.595484 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.595389 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.595921 18444 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.597473 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.599462 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.574932 20228 consensus_queue.cc:1243] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:11.598960 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.587635 20230 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:11.601572 20230 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.602481 20230 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.607401 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.608002 18291 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.608055 20230 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:11.613610 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08002e20a0 after lost signal to thread 18324
I20260709 12:17:11.618382 19040 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:11.615082 19035 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:17:11.626469 20311 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:17:11.632298 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.631417 20228 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:17:11.634253 20228 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.634629 20228 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.635161 20271 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.649998 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.660442 20273 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.670475 20222 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.670955 20222 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:11.672663 18291 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:11.677245 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.678654 18296 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.676512 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.671867 20081 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.682601 20222 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.683328 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:11.683851 20222 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.684065 18292 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 2: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:11.684546 18375 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:11.635407 20228 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.684952 18375 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.685696 18297 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.687594 18375 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.688827 18375 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:11.693295 20228 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.697180 20134 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:11.697672 20134 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.696491 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:11.703532 20134 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:11.707850 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.716812 19040 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.721488 19040 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:11.723189 19040 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.723897 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:11.724202 19039 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.727579 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.729631 18297 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.731729 19038 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:11.734299 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84413.3us
W20260709 12:17:11.734719 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.737399 20218 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.737699 20234 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.739346 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:11.740183 18449 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:11.741074 18449 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.740429 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.741812 18440 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:11.742210 18449 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:11.742798 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.743949 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.744545 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.755259 18449 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.754117 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.759506 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.759673 20032 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:11.760785 20032 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.761029 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.761710 20032 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:11.762070 20032 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.761940 20078 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:11.762656 20127 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.758394 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.765681 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.769871 18297 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.770448 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.770711 18299 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:11.770993 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.773752 18298 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:17:11.788573 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.771791 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.789402 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.790271 18299 raft_consensus.cc:2368] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
W20260709 12:17:11.795176 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.798659 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.770483 20239 raft_consensus.cc:993] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
W20260709 12:17:11.832777 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61865.5us
I20260709 12:17:11.840637 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:11.814978 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:11.823367 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.841840 18449 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.763057 20127 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:11.764079 20271 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.787976 20302 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:11.791335 19971 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.848357 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.791320 20158 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.788174 20283 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.752715 20237 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:17:11.791478 20193 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.791126 20249 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:11.854240 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55714.5us
I20260709 12:17:11.862077 19971 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:11.850296 20271 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:11.798014 20265 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.763861 20234 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:17:11.833930 19853 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:17:11.870790 20273 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.854019 19880 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:11.868037 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.869194 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:11.900988 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.836766 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:11.875433 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.843885 20288 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:17:11.871410 20139 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:17:11.910202 20273 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.911417 20193 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.911888 20193 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:11.912892 20193 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.914151 20193 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.915802 20196 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.915586 20127 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:11.850163 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.871186 20148 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:11.799360 18440 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.888163 20271 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.902240 18367 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:11.921373 18369 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.903291 19880 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.921309 18375 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:11.922633 18366 raft_consensus.cc:606] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:11.923023 18366 raft_consensus.cc:614] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:11.797372 20032 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:11.924880 18373 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1349) took 2277 ms (client timeout 1525 ms). Trace:
W20260709 12:17:11.925359 18373 rpcz_store.cc:269] 0709 12:17:09.644336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.722396 (+ 78060us) service_pool.cc:224] Handling call
0709 12:17:11.921021 (+2198625us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54003f1954. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:11.921124 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953718528}
I20260709 12:17:11.921571 20247 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.921406 20207 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.889437 20265 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:11.892014 20280 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.975634 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 370) took 1082 ms. Trace:
I20260709 12:17:11.975634 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:11.932072 20127 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.980125 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:11.983575 20317 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:17:11.985924 20265 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:11.988090 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.989359 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:11.924434 20148 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:11.990902 20127 raft_consensus.cc:1081] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Signalling peer 991dcd2a70c74fdb89ca9494cc09fb7b to start an election
I20260709 12:17:11.998893 19038 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:11.999511 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:11.994422 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.000991 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:11.993763 20230 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:11.992900 20148 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:11.979151 20076 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:12.008203 18445 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:11.976568 18448 rpcz_store.cc:276] 0709 12:17:10.892870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.893518 (+   648us) service_pool.cc:224] Handling call
0709 12:17:10.897421 (+  3903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.908100 (+ 10679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.908132 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.908176 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:10.908205 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:11.975422 (+1067217us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54000d8254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:11.975463 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:11.975577 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24129920}
I20260709 12:17:12.015313 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:12.017732 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:12.020224 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:11.981629 20247 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:12.049551 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.022524 20020 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:12.050333 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:12.050160 20307 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:12.058274 18299 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:12.052978 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.060453 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.009150 20195 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:12.061864 20295 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:12.021034 20273 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.020871 20134 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:12.056335 20258 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:17:12.043712 20076 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:12.066498 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:12.050940 19853 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.077399 20262 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:12.077952 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 231462us
W20260709 12:17:12.059042 18470 debug-util.cc:398] Leaking SignalData structure 0x7b0800266ca0 after lost signal to thread 18399
I20260709 12:17:12.057622 20222 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:12.094928 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.104720 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 369) took 1215 ms. Trace:
I20260709 12:17:12.098345 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.105195 18446 rpcz_store.cc:276] 0709 12:17:10.889587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.889690 (+   103us) service_pool.cc:224] Handling call
0709 12:17:10.897767 (+  8077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.544822 (+647055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:12.104423 (+559601us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54002ab054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:12.104640 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485452544}
I20260709 12:17:12.112031 18373 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:12.111599 19990 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:17:12.123100 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:12.123862 18449 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:12.107815 20260 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.090430 20134 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:12.096050 20127 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:17:12.193114 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:12.194850 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.196074 20206 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:12.185333 20307 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:17:12.196604 20206 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:12.196962 20280 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:12.197219 19036 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
 from {username='slave'} at 127.0.0.1:59430
I20260709 12:17:12.197482 20206 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.197865 19036 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.198428 19036 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:12.169700 20247 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:12.198676 20206 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:12.095432 19853 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:12.200670 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:12.221186 20285 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:12.226131 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:12.235563 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.264240 18446 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.266183 20264 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:12.197801 20134 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:12.165711 20260 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:12.196973 20307 consensus_queue.cc:1243] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:12.221053 20092 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:12.281783 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.200181 18470 debug-util.cc:398] Leaking SignalData structure 0x7b080035a280 after lost signal to thread 18473
I20260709 12:17:12.197798 20237 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
W20260709 12:17:12.288965 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.314006 18446 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:12.314882 18442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.022s
W20260709 12:17:12.315321 18442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.006s	sys 0.022s
I20260709 12:17:12.129288 20076 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.307852 20260 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.322650 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 363) took 1477 ms. Trace:
I20260709 12:17:12.324108 18444 rpcz_store.cc:276] 0709 12:17:10.844861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.857552 (+ 12691us) service_pool.cc:224] Handling call
0709 12:17:10.857872 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:11.596745 (+738873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:12.322430 (+725685us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54003b1ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:12.322587 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696920960}
I20260709 12:17:12.328307 18446 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:12.324342 20237 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:12.345441 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.364152 20307 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:12.339186 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08003748a0 after lost signal to thread 19068
W20260709 12:17:12.375901 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 329738us
I20260709 12:17:12.363811 20302 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:12.369433 20177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.015s	sys 0.048s
W20260709 12:17:12.385144 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 369907us
I20260709 12:17:12.391007 18366 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.391297 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.393033 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.379522 20327 raft_consensus.cc:993] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
W20260709 12:17:12.384290 18470 debug-util.cc:398] Leaking SignalData structure 0x7b080026f620 after lost signal to thread 20080
I20260709 12:17:12.378901 20237 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:12.400252 18299 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:12.404847 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:12.376657 20307 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.412889 19039 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:12.395761 20134 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.422785 18292 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:12.385965 20177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.015s	sys 0.048s
I20260709 12:17:12.396763 19853 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:12.420465 20327 raft_consensus.cc:1081] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:17:12.390184 20239 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:17:12.396517 20218 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.410279 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.437049 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:12.488376 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 78244.4us
I20260709 12:17:12.489244 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:12.438289 20237 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.398205 20271 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.493714 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:12.468518 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 201318us
I20260709 12:17:12.470866 19853 raft_consensus.cc:993] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:17:12.494998 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:12.495723 18300 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.496099 18300 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:12.415371 20322 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:17:12.427057 20049 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:17:12.501741 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.497786 20225 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.500967 20020 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:12.398248 20153 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:12.480188 20127 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.427215 20273 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 12:17:12.523367 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.525631 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.511382 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:12.548565 18292 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:12.497938 20222 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.501351 20283 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.501633 20323 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 12:17:12.559321 20330 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:17:12.559401 20321 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:17:12.561923 20285 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.563949 20271 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:12.557435 20153 consensus_queue.cc:1243] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:17:12.571390 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 392519us
I20260709 12:17:12.469640 19036 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.583395 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.548251 20127 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:12.599731 20328 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:17:12.601454 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 383) took 1344 ms. Trace:
I20260709 12:17:12.601902 18442 rpcz_store.cc:276] 0709 12:17:11.257241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.298555 (+ 41314us) service_pool.cc:224] Handling call
0709 12:17:11.298952 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:11.299963 (+  1011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:11.300117 (+   154us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:11.740622 (+440505us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:11.740978 (+   356us) log.cc:844] Serialized 26 byte log entry
0709 12:17:12.315746 (+574768us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:12.315799 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:12.315951 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:12.601105 (+285154us) raft_consensus.cc:1685] finished
0709 12:17:12.601130 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:12.601373 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":62380416,"thread_start_us":574369,"threads_started":1,"wal-append.queue_time_us":648367}
I20260709 12:17:12.607124 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:12.608932 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.611613 19853 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.580212 20327 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:17:12.613884 19853 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.596305 20217 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:12.618621 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:12.619915 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:12.607424 19971 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.620675 20217 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:12.623801 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1883) took 2611 ms. Trace:
I20260709 12:17:12.613840 20326 raft_consensus.cc:1081] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:17:12.624648 19853 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:12.657101 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 155452us
W20260709 12:17:12.624110 20331 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.181s	user 0.003s	sys 0.000s
I20260709 12:17:12.646019 18441 rpcz_store.cc:276] 0709 12:17:10.012077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.012860 (+   783us) service_pool.cc:224] Handling call
0709 12:17:10.015946 (+  3086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:10.016280 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.016314 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:11.739797 (+1723483us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:11.740170 (+   373us) log.cc:844] Serialized 26 byte log entry
0709 12:17:12.005357 (+265187us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:12.005407 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:12.005499 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:12.620049 (+614550us) raft_consensus.cc:1685] finished
0709 12:17:12.620068 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:12.623681 (+  3613us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":372,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6361,"spinlock_wait_cycles":3014120192,"thread_start_us":259306,"threads_started":1,"wal-append.queue_time_us":574476}
I20260709 12:17:12.781931 18442 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:12.667639 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 180095us
I20260709 12:17:12.624017 20077 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:12.689198 20335 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:12.625752 20222 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:12.685637 20193 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.771s	user 0.014s	sys 0.001s
I20260709 12:17:12.652107 20217 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.696977 20127 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:12.786402 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.786027 19853 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.817445 19853 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.795713 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.801206 18441 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:12.817935 19853 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:12.821270 20341 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.821831 20341 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.776028 19971 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:12.827301 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.785264 20307 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:12.785506 20077 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:12.843748 20341 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:12.845678 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.857498 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.860193 20158 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:12.830186 20206 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:12.832712 20139 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.891749 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:12.892035 18446 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:12.901156 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:12.904639 20077 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:12.903367 19036 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:12.909693 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1904) took 1165 ms. Trace:
I20260709 12:17:12.910117 18445 rpcz_store.cc:276] 0709 12:17:11.744045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.744616 (+   571us) service_pool.cc:224] Handling call
0709 12:17:11.744836 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:12.048329 (+303493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:12.909417 (+861088us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:12.909623 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605103872}
I20260709 12:17:12.910612 18447 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:17:12.913424 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 320178us
I20260709 12:17:12.891155 20158 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:12.916054 18447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:17:12.916201 18445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48158
I20260709 12:17:12.916960 18447 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.917057 18445 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.917546 18447 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:12.917634 18445 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:12.920077 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.919344 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 79119.7us
I20260709 12:17:12.921772 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:12.921862 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:12.922485 18372 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:12.922952 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:12.926000 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:12.929445 20351 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:12.930533 20351 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:12.930902 20351 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:12.931603 20351 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:12.936288 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:12.938488 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 106493us
I20260709 12:17:12.903858 20206 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:12.940160 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.947751 18448 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:12.948932 20350 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.935976 20151 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:12.954878 20151 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.955670 20081 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:12.957228 20151 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:12.959962 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:12.968199 18447 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:12.971776 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:12.972029 18447 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:12.972420 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:12.974442 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:12.975271 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:12.976504 18338 leader_election.cc:400] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:12.980636 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:12.981269 18298 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:12.981406 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:12.982084 18292 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:12.982816 20271 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:12.983680 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.984750 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:12.970552 20350 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.986423 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:12.989261 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:12.968909 20153 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:12.999766 20286 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.974480 20139 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.000638 20139 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.005650 18449 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.006071 20260 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.000s
I20260709 12:17:13.006503 20260 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:12.988188 20346 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:13.008497 20346 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.008882 20346 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.008797 20153 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.009421 18441 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.984337 20206 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:12.988567 20350 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.002926 20139 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.014415 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.025190 18447 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.025617 18442 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:13.006585 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1480) took 1121 ms. Trace:
I20260709 12:17:13.018976 18445 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.022881 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:13.029222 18449 rpcz_store.cc:276] 0709 12:17:11.885412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.319419 (+434007us) service_pool.cc:224] Handling call
0709 12:17:13.005276 (+685857us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57eee 000000000036eb39 00007f3c6ddcb1e0 00007f3c6de940c4 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:13.005440 (+   164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.006293 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:13.006515 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3870976}
I20260709 12:17:13.029587 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:12.975839 20353 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.030967 18441 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.026624 20295 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.943s	user 0.004s	sys 0.008s
I20260709 12:17:12.982880 20333 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:13.024987 20350 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.011109 20153 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.060454 19038 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.061682 20356 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.062306 20356 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.063225 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.039799 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.042191 18445 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:12.999043 20217 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.070544 20348 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:17:13.035391 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.035465 20325 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.040223 20353 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.080403 20260 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.081338 20217 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.082259 20217 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:13.058534 20295 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:13.059816 20268 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.089052 20345 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:13.054510 20307 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:13.089628 20345 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.090109 20345 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:13.054883 20327 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:17:13.091043 20327 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:13.091399 20327 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:13.092226 20327 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.093561 20327 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:13.073486 20353 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.072068 20153 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.086400 19036 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:13.087100 20217 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:17:13.108245 20134 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.109323 20359 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:13.084712 20325 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.108902 20153 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.114679 20307 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.104835 20268 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.126181 20359 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.128489 20359 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:13.122027 20134 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.132606 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.117340 20333 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.137526 20268 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.138397 19853 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.139405 20265 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:13.141718 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1482) took 1253 ms. Trace:
I20260709 12:17:13.142095 18448 rpcz_store.cc:276] 0709 12:17:11.887498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.050051 (+162553us) service_pool.cc:224] Handling call
0709 12:17:13.140541 (+1090490us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54002a9c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:13.140646 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753011072}
I20260709 12:17:13.143122 18440 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
W20260709 12:17:13.154695 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 84992.8us
I20260709 12:17:13.143699 20249 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.157822 18448 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.157475 20249 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.159101 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.140419 20307 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.165330 20307 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.122771 20153 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.131122 20134 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.175132 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:13.175773 18445 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.183504 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:13.183904 18447 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.185325 20345 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.166841 20268 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:17:13.190032 20158 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.193482 20353 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.194454 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1485) took 1253 ms. Trace:
I20260709 12:17:13.194836 18443 rpcz_store.cc:276] 0709 12:17:11.940907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.989135 (+ 48228us) service_pool.cc:224] Handling call
0709 12:17:13.194244 (+1205109us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b5400424354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:13.194367 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338323840}
I20260709 12:17:13.200925 18443 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:13.202220 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 83519.7us
I20260709 12:17:13.206465 20353 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.206638 19036 raft_consensus.cc:2368] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3.
I20260709 12:17:13.201669 20158 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.172446 20325 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.203634 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:13.210180 18445 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:13.177430 20265 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.197269 20134 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.143910 20333 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.204553 20346 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.237672 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.239883 20346 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.240494 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.241199 18375 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:13.232256 20362 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:17:13.243798 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.245471 18366 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:13.247118 18366 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:13.247067 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:13.233039 20338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.012s	sys 0.018s
I20260709 12:17:13.282174 20346 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.233289 20134 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.234402 20195 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.284533 19035 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:13.284965 20367 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:13.285180 18448 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.285521 20371 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:13.289669 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.236901 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.284909 18368 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:13.321237 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.322182 19043 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:13.322002 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.299981 20345 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.324141 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.312438 20195 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.326653 20076 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.313931 20346 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.329480 20346 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:13.330446 20295 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.304940 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:13.307852 18375 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:13.331983 18443 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:13.332525 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.285542 20333 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.280020 20327 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:17:13.344895 18292 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:13.318058 20379 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:13.308847 20338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.013s	sys 0.018s
I20260709 12:17:13.309478 20307 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.334059 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.284008 18374 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:13.335062 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:13.337605 20153 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.333797 20367 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:13.232640 20265 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.281769 20222 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.331413 18370 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.341028 20286 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:13.313172 19990 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:17:13.342234 20374 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.343422 18443 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:13.331027 20195 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.348394 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:13.372992 18297 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.351529 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:13.373850 18291 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:13.333480 20325 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:13.347954 20230 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.354619 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:13.356612 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:13.358062 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.389340 18366 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.361289 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.352988 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.331081 20295 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.360598 20359 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:13.389971 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:13.390928 18296 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:13.391537 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:13.391866 18297 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:13.392371 20230 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.393186 20215 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:13.372361 20371 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.393945 20340 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.393959 20134 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.363125 20345 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.356184 20307 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.398804 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:13.393880 20334 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.732s	user 0.003s	sys 0.000s
I20260709 12:17:13.395130 20193 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:13.406144 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.365195 20222 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.400941 20271 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:13.416015 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.416810 18374 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.417155 18374 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:13.402920 20215 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:13.417287 20380 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.417971 18374 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.419999 18374 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.406131 20359 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:13.355602 20379 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.397559 20276 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:17:13.408668 20340 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:13.421084 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.401034 20286 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:13.419786 20380 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.429826 20230 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.430908 20380 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.437455 18291 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:13.437990 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.390209 20374 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.447395 20374 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.449446 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.452402 18295 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:13.428243 19990 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:13.413434 20345 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.456094 18371 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.438879 20158 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.453930 20374 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.458503 20383 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.423633 20300 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:17:13.409813 20371 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.441198 20333 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:13.473517 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82082.5us
I20260709 12:17:13.443202 20380 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.420493 20367 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.464218 19990 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:13.484083 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.478499 20359 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.506294 18374 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:13.507401 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.479002 20333 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.508109 18374 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:13.508999 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.509562 18374 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:13.512451 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:13.518561 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.519202 18448 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:13.519371 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:13.520144 18292 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.510094 20384 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.526865 19041 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:13.474043 20374 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.534767 20222 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.535534 20222 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.536157 20222 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:13.527648 20376 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:17:13.501766 20081 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.528123 20195 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.528805 20265 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.529978 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.550886 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:13.555981 18448 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:13.552414 20345 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.552735 20222 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.467932 20215 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.556058 18296 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.514889 20153 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.473384 20158 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:13.559446 20382 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 12:17:13.560660 20049 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.564006 20081 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.564525 20371 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.565073 20295 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:13.572435 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:13.540634 20350 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.599938 18298 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.479944 19990 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:13.572211 20391 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.542523 20286 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.479321 20380 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.575819 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.480312 20383 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.584609 20195 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.512240 20121 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:17:13.596850 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:13.597857 20215 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.585554 20222 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.583636 20333 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.460131 20340 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.516603 20276 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:17:13.566290 20353 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.606277 20049 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.635659 20353 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.576305 20282 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.637427 20282 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.608129 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:13.642375 18448 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:13.642774 18448 pending_rounds.cc:85] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 1
I20260709 12:17:13.610458 18445 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:13.613125 18369 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.614324 20195 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.614987 20081 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.605975 20307 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.585076 20379 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.621093 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.650451 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:13.622385 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:13.650458 20343 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.039s	user 0.001s	sys 0.001s
I20260709 12:17:13.623508 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.620153 20286 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:13.652639 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1474) took 1780 ms (client timeout 1786 ms). Trace:
I20260709 12:17:13.652997 18445 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:13.653055 18440 rpcz_store.cc:269] 0709 12:17:11.872744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:11.982643 (+109899us) service_pool.cc:224] Handling call
0709 12:17:13.652486 (+1669843us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400423e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:13.652587 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506018176}
I20260709 12:17:13.653216 18444 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:13.644995 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.645776 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:13.638314 20076 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.648970 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.623464 20153 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.656924 19038 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:13.659813 20081 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:13.660133 20195 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:13.649307 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:13.667802 18298 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:13.668478 18292 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:13.669394 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.669236 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:13.670332 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.670240 18298 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.658305 18448 pending_rounds.cc:107] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.2
I20260709 12:17:13.615283 20391 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.673704 18448 raft_consensus.cc:2917] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Aborting config change with OpId 1.2: Aborted: Op aborted by new leader
I20260709 12:17:13.674118 18448 raft_consensus.cc:2889] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:13.661532 20158 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:13.626421 20222 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.664304 20049 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.664817 20230 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:13.686954 19035 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:13.700119 18292 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:13.662216 20076 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.690764 18296 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:13.574270 20384 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.700733 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:13.664016 20153 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.705504 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.708216 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.703181 18298 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:13.709512 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.697996 20350 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.724924 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46944 (request call id 10) took 10194 ms. Trace:
I20260709 12:17:13.725227 20049 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.725847 20049 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.725700 18296 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:13.725402 18448 rpcz_store.cc:276] 0709 12:17:03.530855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:03.547733 (+ 16878us) service_pool.cc:224] Handling call
0709 12:17:03.567803 (+ 20070us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de561c0 00007f3c6de429d9 00007f3c6de42ca2 00007f3c673c11d4
0709 12:17:03.802265 (+234462us) log.cc:844] Serialized 506 byte log entry
0709 12:17:13.707131 (+9904866us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25039,"raft.run_cpu_time_us":2876,"raft.run_wall_time_us":2874,"spinlock_wait_cycles":6133504,"thread_start_us":178182,"threads_started":1,"wal-append.queue_time_us":96915,"wal-append.run_cpu_time_us":1999,"wal-append.run_wall_time_us":1013310}
I20260709 12:17:13.728119 20049 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.727103 20282 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.729382 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.724906 20081 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:13.730325 18296 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:13.732208 18296 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:13.732872 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:13.734205 18296 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:13.734570 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:13.735675 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.736284 18296 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:13.737151 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:13.737284 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.731518 20081 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.739038 20195 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:13.739609 20195 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:13.740028 20222 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.731201 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.740475 20195 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.741806 20307 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:13.744872 20397 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:13.745483 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.706306 20380 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.749011 18296 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:13.749643 18448 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.734324 20379 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.753906 20379 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.750115 20384 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.757899 20379 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.750627 20049 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:13.748277 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.698421 20230 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.758479 20139 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:13.750409 20076 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:13.747786 20389 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:13.763587 20397 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:13.763913 20324 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:13.735181 20286 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.767741 20345 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.768086 18372 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:13.771811 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:13.772509 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.786056 19043 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:13.774292 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.738189 20153 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.773252 20285 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:13.773275 20139 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:13.735422 20383 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:13.746665 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.783594 18372 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:13.775964 20322 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.198s	user 0.026s	sys 0.011s
W20260709 12:17:13.790262 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43681.6us
I20260709 12:17:13.787079 20215 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.794934 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.795030 20215 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.795675 20383 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:13.796051 20383 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.796422 20383 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:13.784904 20049 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:13.798341 18372 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.787411 20324 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:13.793627 20322 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.026s	sys 0.011s
I20260709 12:17:13.801916 18372 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.790429 20285 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.800841 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:13.785517 20397 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:13.792829 20153 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.792062 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:13.798062 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.787667 20345 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:13.802110 20237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.007s	sys 0.006s
I20260709 12:17:13.825451 20153 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.819056 20285 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:13.825259 20076 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:13.840380 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 434) took 1439 ms. Trace:
W20260709 12:17:13.831786 20237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.007s	sys 0.007s
I20260709 12:17:13.840785 19041 rpcz_store.cc:276] 0709 12:17:12.401138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.401266 (+   128us) service_pool.cc:224] Handling call
0709 12:17:12.418126 (+ 16860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.532711 (+1114585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:13.840132 (+307421us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54008205d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:13.840295 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548996352}
I20260709 12:17:13.833418 20153 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.844573 20391 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:13.846771 18368 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:13.847127 18368 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:13.848169 20049 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.849900 20077 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.852715 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:13.857247 18372 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:13.857862 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:13.859071 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:13.861850 20177 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:13.818898 20300 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:17:13.896607 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.897229 19043 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:13.894418 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:13.834890 20076 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.899863 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.899271 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:13.899546 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44353.9us
I20260709 12:17:13.830682 20283 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 12:17:13.902699 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.862660 20237 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.891180 20391 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:13.904844 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.896581 20271 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.905783 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:13.908543 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:13.902027 20076 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.903542 20333 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:13.827734 20139 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.901218 20121 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:13.890591 20276 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:17:13.918040 20386 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:13.904487 20379 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:13.918637 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.915143 20195 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.924387 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:13.910800 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:13.920141 20358 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:13.925455 20361 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 12:17:13.934728 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.904368 19990 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.921797 20121 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.913717 20271 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:13.939644 20264 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:13.909946 20237 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.917309 20333 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:13.909412 20391 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:14.000074 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.929965 20139 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.941857 19035 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:13.987623 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.988206 19043 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:13.932672 20195 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.948758 20358 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.014264 18445 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:13.950745 20121 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:13.989864 20264 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.985221 20300 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:13.934930 20379 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:13.911270 20397 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.004145 20333 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:14.008224 20295 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:14.006179 20268 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.014161 20195 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.018239 20358 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:14.013227 20286 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:14.019972 20392 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:14.004290 20367 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:17:13.910902 20324 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:14.024403 20300 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.143078 20276 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.028179 20295 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:14.029731 20268 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:14.221156 20268 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:14.219102 20295 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:14.036192 20286 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.145206 20324 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:14.142731 20049 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:14.256340 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:14.297108 20237 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:14.297904 20237 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:14.298494 20237 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.344820 20391 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:14.346581 20153 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:17:14.347206 20153 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:14.356806 20401 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:17:14.358256 20153 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:14.359009 20406 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:17:14.361114 18296 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:14.361285 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1391) took 1045 ms. Trace:
I20260709 12:17:14.224021 20276 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:14.361792 18374 rpcz_store.cc:276] 0709 12:17:13.316124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.515998 (+199874us) service_pool.cc:224] Handling call
0709 12:17:13.516219 (+   221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:13.516424 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:13.516440 (+    16us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:13.554474 (+ 38034us) log.cc:844] Serialized 26 byte log entry
0709 12:17:13.783195 (+228721us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:13.783278 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:13.783341 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:14.361017 (+577676us) raft_consensus.cc:1685] finished
0709 12:17:14.361044 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:14.361217 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13094,"raft.queue_time_us":52694,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":184039,"threads_started":1,"wal-append.queue_time_us":243155}
W20260709 12:17:14.325605 20405 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 12:17:14.374334 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.142536 20387 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.429s
W20260709 12:17:14.400432 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.354480 19990 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:14.100459 20392 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.306845 20049 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:14.427407 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.395939 20271 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:14.451833 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:14.438414 20379 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:14.437129 20153 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.459146 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1387) took 1205 ms. Trace:
I20260709 12:17:14.461175 18369 rpcz_store.cc:276] 0709 12:17:13.251491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.251613 (+   122us) service_pool.cc:224] Handling call
0709 12:17:13.251836 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.643729 (+391893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.457085 (+813356us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54003f2354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:14.457293 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830458752}
I20260709 12:17:14.444339 20271 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.462389 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.464851 18369 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:14.466002 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.466761 18369 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
W20260709 12:17:14.468183 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.052s	sys 0.112s
W20260709 12:17:14.469165 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.052s	sys 0.112s
I20260709 12:17:14.470633 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.473114 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:14.480829 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.480755 18369 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:14.482089 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.483413 18369 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:14.484107 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:14.464500 20407 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.432s	user 0.000s	sys 0.001s
I20260709 12:17:14.492058 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1906) took 2420 ms. Trace:
I20260709 12:17:14.478077 20148 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.492555 18446 rpcz_store.cc:276] 0709 12:17:12.071814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.329216 (+257402us) service_pool.cc:224] Handling call
0709 12:17:12.365154 (+ 35938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:12.898875 (+533721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.491793 (+1592918us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:14.491983 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211074816}
I20260709 12:17:14.483173 20078 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:14.509689 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.513002 18369 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:14.489321 20359 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:14.486176 20351 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.001s	sys 0.014s
I20260709 12:17:14.501011 20295 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:14.509673 20402 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 12:17:14.472160 20271 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:14.529414 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.023s	sys 0.022s
I20260709 12:17:14.492842 20300 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:14.507944 20286 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:14.461750 20386 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.001s	sys 0.017s
I20260709 12:17:14.469723 20153 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:13.931310 20158 consensus_queue.cc:1175] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e23c0ce8d89940da8eac6a6002e349c5" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:15.096556 18298 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:14.708659 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.115061 18370 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:14.754437 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:14.755681 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1918) took 1812 ms. Trace:
I20260709 12:17:14.728147 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1486) took 2729 ms. Trace:
I20260709 12:17:14.555857 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:14.549237 18340 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.025s
W20260709 12:17:15.120005 18415 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.842s (SENT) [suppressed 21 similar messages]
I20260709 12:17:14.533659 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:14.517875 20078 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:15.122480 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet ca383fa1243547c3b32dfbd6857a3fba: Aborted: Op aborted by new leader
I20260709 12:17:14.699245 20207 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:15.123553 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.035s	sys 0.030s
W20260709 12:17:14.727063 20077 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.877s	user 0.007s	sys 0.003s
W20260709 12:17:14.729421 20121 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.000s
I20260709 12:17:15.124420 20134 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:14.789098 20139 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.640s	user 0.003s	sys 0.006s
I20260709 12:17:14.515386 20148 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.700707 20206 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.426s
W20260709 12:17:14.641245 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:14.756994 19036 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 442) took 1611 ms (client timeout 1694 ms). Trace:
W20260709 12:17:14.743438 19035 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.722s	user 0.005s	sys 0.005s
I20260709 12:17:15.129490 20032 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:14.529064 20359 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.183389 20356 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.184492 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1539) took 1440 ms. Trace:
I20260709 12:17:15.184940 18370 rpcz_store.cc:276] 0709 12:17:13.743760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:14.708320 (+964560us) service_pool.cc:224] Handling call
0709 12:17:15.184278 (+475958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.185802 20359 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:14.654794 20076 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.733s	user 0.007s	sys 0.000s
I20260709 12:17:15.130290 20153 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:14.793494 20283 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.077s	sys 0.184s
I20260709 12:17:14.560545 20237 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.131541 20225 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:14.774935 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1507) took 5048 ms. Trace:
W20260709 12:17:15.068104 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.153612 20386 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:14.740732 20333 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.002s
I20260709 12:17:15.186966 20286 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:14.804522 20361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.011s	sys 0.008s
I20260709 12:17:15.155812 18441 rpcz_store.cc:276] 0709 12:17:12.943255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.010196 (+ 66941us) service_pool.cc:224] Handling call
0709 12:17:13.010468 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.038483 (+ 28015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.755373 (+1716890us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b54001a5254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:14.755601 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46638848}
W20260709 12:17:14.753357 20358 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.000s
W20260709 12:17:14.744663 20285 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.882s	user 0.010s	sys 0.001s
I20260709 12:17:14.724406 20345 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.158802 20353 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.158525 20295 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:15.159863 20351 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.005s	sys 0.042s
I20260709 12:17:15.160125 20078 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.158567 20207 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:15.161159 20390 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:14.666268 20383 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.869s	user 0.007s	sys 0.002s
I20260709 12:17:14.786474 20215 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:14.592180 20276 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.172060 20121 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.130754 20049 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:15.173013 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.222741 20258 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:15.223484 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.222929 20333 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:14.832974 20195 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.000s
I20260709 12:17:15.223691 20258 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.224885 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.225126 20195 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.225759 18299 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:15.225903 20333 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.226251 20258 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.193193 20302 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.157104 18444 rpcz_store.cc:276] 0709 12:17:11.999035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.325254 (+326219us) service_pool.cc:224] Handling call
0709 12:17:13.652898 (+1327644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.653992 (+  1094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.727890 (+1073898us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400423e50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:14.728066 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3052592896}
W20260709 12:17:14.741940 20388 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.128s	user 0.030s	sys 0.011s
I20260709 12:17:15.226984 20225 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.227072 20358 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:15.194674 20283 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.086s	sys 0.186s
W20260709 12:17:14.788770 20264 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.764s	user 0.007s	sys 0.002s
I20260709 12:17:15.230262 20358 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.184958 20230 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.205353 20260 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:15.206137 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.016s	sys 0.054s
I20260709 12:17:14.766543 20374 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:15.231312 20361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.016s	sys 0.012s
I20260709 12:17:15.233218 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 444) took 2034 ms. Trace:
I20260709 12:17:15.235033 18300 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:15.236512 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.208516 20206 consensus_queue.cc:1243] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:15.218680 20256 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.218916 19035 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:15.193113 20076 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.170886 20134 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:14.876243 20249 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.226310 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1464) took 1369 ms. Trace:
I20260709 12:17:15.227571 20195 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.195066 18293 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:15.230048 20225 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:14.709671 20344 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:15.215171 20153 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.193464 18371 rpcz_store.cc:276] 0709 12:17:09.726377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:10.909478 (+1183101us) service_pool.cc:224] Handling call
0709 12:17:11.166238 (+256760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.461346 (+2295108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.774042 (+1312696us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54000c9ed4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:14.774688 (+   646us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54000c9ed0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:14.774857 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5843276928}
I20260709 12:17:15.204612 20392 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.236439 18300 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.187196 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.236855 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 296) took 1578 ms. Trace:
I20260709 12:17:15.238031 20359 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.218921 20300 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.242031 18299 rpcz_store.cc:276] 0709 12:17:13.856425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.224176 (+1367751us) service_pool.cc:224] Handling call
0709 12:17:15.226217 (+  2041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.251478 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1540) took 1487 ms. Trace:
I20260709 12:17:15.247138 18294 rpcz_store.cc:276] 0709 12:17:13.658156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.753347 (+ 95191us) service_pool.cc:224] Handling call
0709 12:17:13.753595 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.755085 (+  1490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:13.755114 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:13.755166 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:13.755199 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.236333 (+1481134us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b54003d9d54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.236407 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.236632 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2962688}
I20260709 12:17:15.254573 20300 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:15.249269 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 689256us
I20260709 12:17:15.256389 20207 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.257547 20300 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.258103 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:15.258841 20283 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.254139 20273 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.252070 18369 rpcz_store.cc:276] 0709 12:17:13.744033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:14.513998 (+769965us) service_pool.cc:224] Handling call
0709 12:17:15.220824 (+706826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.259054 18371 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.258301 20207 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:15.260032 20283 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.263418 20344 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:15.263058 20286 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.265237 20283 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.265100 20344 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.265754 20286 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.266579 20286 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.267181 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1869) took 2288 ms. Trace:
I20260709 12:17:15.249435 20383 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.267379 20322 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.268150 20386 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.269737 20338 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:15.270030 20134 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.249810 20302 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.231035 20264 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.271660 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1462) took 1415 ms. Trace:
I20260709 12:17:15.272915 20076 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.274164 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1384) took 2025 ms. Trace:
I20260709 12:17:15.274235 19044 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:15.244488 19036 rpcz_store.cc:269] 0709 12:17:13.145715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.155581 (+  9866us) service_pool.cc:224] Handling call
0709 12:17:14.756742 (+1601161us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b540066e4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:14.756915 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24567168}
I20260709 12:17:15.300164 20338 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:15.275272 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.309039 20121 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.276311 19853 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.277529 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.310034 18369 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:15.310171 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.310515 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 294) took 1691 ms. Trace:
I20260709 12:17:15.310931 18369 rpcz_store.cc:276] 0709 12:17:13.619495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.277077 (+1657582us) service_pool.cc:224] Handling call
0709 12:17:15.310426 (+ 33349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.278255 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 289) took 2069 ms. Trace:
I20260709 12:17:15.276355 20148 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.260241 20273 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:15.268214 19035 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 445) took 1409 ms (client timeout 1824 ms). Trace:
I20260709 12:17:15.280781 20356 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:15.281854 18368 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.233709 19037 rpcz_store.cc:276] 0709 12:17:13.198589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.205234 (+  6645us) service_pool.cc:224] Handling call
0709 12:17:13.231767 (+ 26533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:13.242542 (+ 10775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:13.279978 (+ 37436us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:13.296780 (+ 16802us) log.cc:844] Serialized 26 byte log entry
0709 12:17:14.750506 (+1453726us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b18001db5a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:17:14.972670 (+222164us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:14.972775 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.157763 (+184988us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:15.231090 (+ 73327us) raft_consensus.cc:1685] finished
0709 12:17:15.231112 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.232926 (+  1814us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1108,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":37810,"spinlock_wait_cycles":10836224,"thread_start_us":221781,"threads_started":1,"wal-append.queue_time_us":1931520}
W20260709 12:17:15.283496 18295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1882) took 1927 ms (client timeout 1799 ms). Trace:
I20260709 12:17:15.283715 20295 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.272830 19041 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:15.285972 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.244658 20390 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.267020 20344 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.301371 20256 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.335543 20273 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.336071 20387 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.336890 20256 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.336758 20387 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.338066 20264 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.339190 20387 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.332603 20153 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.343776 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.345697 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.346107 20153 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.299417 18291 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:15.309216 19043 raft_consensus.cc:2368] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
W20260709 12:17:15.278445 18292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1458) took 1796 ms (client timeout 1921 ms). Trace:
I20260709 12:17:15.313107 19039 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:15.311060 19040 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:15.275290 20160 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:15.298923 20285 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.311795 18367 rpcz_store.cc:276] 0709 12:17:13.209116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.209286 (+   170us) service_pool.cc:224] Handling call
0709 12:17:13.209617 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.385612 (+175995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:13.385641 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:13.385682 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:13.385729 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.277812 (+1892083us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.277878 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.278070 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404258688}
I20260709 12:17:15.304193 18296 rpcz_store.cc:276] 0709 12:17:13.856023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.857624 (+  1601us) service_pool.cc:224] Handling call
0709 12:17:15.271180 (+1413556us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400330254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.271466 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155385600}
I20260709 12:17:15.306663 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1542) took 1558 ms. Trace:
I20260709 12:17:15.313719 20134 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.324438 20284 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.324608 20302 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.325081 20322 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.307156 18366 rpcz_store.cc:276] 0709 12:17:13.248072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.248162 (+    90us) service_pool.cc:224] Handling call
0709 12:17:13.334613 (+ 86451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.389713 (+ 55100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.272340 (+1882627us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b5400395654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.273689 (+  1349us) spinlock_profiling.cc:243] Waited 86.2 ms on lock 0x7b5400395650. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.273910 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323725568}
I20260709 12:17:15.359965 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.360203 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 289) took 1878 ms. Trace:
I20260709 12:17:15.360985 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1541) took 1616 ms. Trace:
I20260709 12:17:15.361410 18366 rpcz_store.cc:276] 0709 12:17:13.744309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.359610 (+1615301us) service_pool.cc:224] Handling call
0709 12:17:15.360907 (+  1297us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:15.362197 18374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1401) took 1698 ms (client timeout 1973 ms). Trace:
I20260709 12:17:15.362514 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:15.362685 18374 rpcz_store.cc:269] 0709 12:17:13.664161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:14.373758 (+709597us) service_pool.cc:224] Handling call
0709 12:17:15.361892 (+988134us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54000c8fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.362034 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866629504}
I20260709 12:17:15.296131 20225 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.363562 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.364287 18374 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:15.334281 20425 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:15.329986 20356 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:15.330531 18295 rpcz_store.cc:269] 0709 12:17:13.356200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.356327 (+   127us) service_pool.cc:224] Handling call
0709 12:17:15.283085 (+1926758us) spinlock_profiling.cc:243] Waited 63 ms on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.283287 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144993664}
I20260709 12:17:15.343926 19037 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:15.294708 18293 rpcz_store.cc:276] 0709 12:17:12.978629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.978757 (+   128us) service_pool.cc:224] Handling call
0709 12:17:12.981240 (+  2483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.245043 (+2263803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.266714 (+ 21671us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54001cbbd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.267022 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5089619712}
W20260709 12:17:15.316668 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.286s	user 0.022s	sys 0.017s
W20260709 12:17:15.318694 19035 rpcz_store.cc:269] 0709 12:17:13.858854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.862228 (+  3374us) service_pool.cc:224] Handling call
0709 12:17:15.261190 (+1398962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.346520 19042 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
W20260709 12:17:15.347306 18292 rpcz_store.cc:269] 0709 12:17:13.481635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.669177 (+187542us) service_pool.cc:224] Handling call
0709 12:17:15.278095 (+1608918us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b540040c754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.278266 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69152000}
I20260709 12:17:15.353607 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.353753 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.354465 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.350994 18370 rpcz_store.cc:276] 0709 12:17:13.744549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.186495 (+1441946us) service_pool.cc:224] Handling call
0709 12:17:15.299734 (+113239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.354754 20127 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:15.352363 20284 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:15.355903 20285 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.374517 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1546) took 1060 ms. Trace:
I20260709 12:17:15.375856 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.402557 18366 rpcz_store.cc:276] 0709 12:17:14.311014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.362166 (+1051152us) service_pool.cc:224] Handling call
0709 12:17:15.371633 (+  9467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.356464 20383 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.365712 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1531) took 1987 ms. Trace:
I20260709 12:17:15.366587 20271 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.334132 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.258329 20260 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.363900 20225 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:15.360731 18298 rpcz_store.cc:276] 0709 12:17:13.482210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.732708 (+250498us) service_pool.cc:224] Handling call
0709 12:17:13.733049 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.112975 (+1379926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.359920 (+246945us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002760d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.360134 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3134155008}
I20260709 12:17:15.380961 18371 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:17:15.383356 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.022s	sys 0.017s
I20260709 12:17:15.387253 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 291) took 2048 ms. Trace:
I20260709 12:17:15.411744 18374 rpcz_store.cc:276] 0709 12:17:13.327653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.363624 (+2035971us) service_pool.cc:224] Handling call
0709 12:17:15.364030 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.373154 (+  9124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.375496 (+  2342us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:15.405227 20020 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:17:15.353543 20134 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.349195 20286 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.372696 20356 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:15.406039 18369 rpcz_store.cc:276] 0709 12:17:13.378002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.364559 (+1986557us) service_pool.cc:224] Handling call
0709 12:17:15.365007 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.365369 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.365394 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.365450 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:15.365498 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.365549 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.365644 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.347447 20359 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.367297 20322 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.416074 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1548) took 1036 ms. Trace:
I20260709 12:17:15.422264 18371 rpcz_store.cc:276] 0709 12:17:14.375117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.255132 (+880015us) service_pool.cc:224] Handling call
0709 12:17:15.411496 (+156364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.382952 20127 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:15.415045 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1868) took 2919 ms. Trace:
I20260709 12:17:15.372210 20153 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.423458 18300 rpcz_store.cc:276] 0709 12:17:12.495197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.495327 (+   130us) service_pool.cc:224] Handling call
0709 12:17:12.495615 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.234069 (+2738454us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b4c003878c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6711f644 00007f3c67146f2d 00007f3c67146b93 00007f3c68e7718b 00007f3c68e77cb5 00007f3c68e72f30 00007f3c68e747de 00007f3c68e7eb49 00007f3c68e7f416
0709 12:17:15.373674 (+139605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.373699 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.373742 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:15.373778 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.382650 (+  8872us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.414958 (+ 32308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119877504}
I20260709 12:17:15.422295 18368 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:15.429569 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.430043 20127 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.431489 18373 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:15.431818 20127 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:15.432304 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 293) took 1899 ms. Trace:
I20260709 12:17:15.432758 18371 rpcz_store.cc:276] 0709 12:17:13.532572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.423126 (+1890554us) service_pool.cc:224] Handling call
0709 12:17:15.431488 (+  8362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.431883 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.431920 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.431980 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:15.432027 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.432099 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.432241 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.436578 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.437331 19036 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:15.309067 20338 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.425148 19040 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:15.425652 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:15.438933 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:15.441155 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 321370us
I20260709 12:17:15.449415 19043 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:15.425457 20427 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:15.444264 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.445658 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.444321 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1388) took 2192 ms. Trace:
W20260709 12:17:15.419059 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.48878e+06us
I20260709 12:17:15.450150 19041 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:15.353844 20160 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.455132 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.455857 18370 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:15.455946 20077 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:15.455883 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54936.8us
I20260709 12:17:15.482527 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.458438 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.461194 20260 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.464897 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.464140 20284 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.486537 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.487304 20284 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.488123 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.468676 20340 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.492568 18297 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:15.493018 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1465) took 1632 ms. Trace:
I20260709 12:17:15.493283 18297 rpcz_store.cc:276] 0709 12:17:13.860526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.457882 (+1597356us) service_pool.cc:224] Handling call
0709 12:17:15.492953 (+ 35071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.494192 18299 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:15.494694 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1463) took 1638 ms. Trace:
I20260709 12:17:15.494953 18299 rpcz_store.cc:276] 0709 12:17:13.856228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.402371 (+1546143us) service_pool.cc:224] Handling call
0709 12:17:15.494623 (+ 92252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.468829 20225 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.456344 18367 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:15.497928 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1545) took 1671 ms. Trace:
I20260709 12:17:15.498284 18367 rpcz_store.cc:276] 0709 12:17:13.826796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.371927 (+1545131us) service_pool.cc:224] Handling call
0709 12:17:15.497872 (+125945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.498203 20077 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.475467 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.477041 20286 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:15.468858 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1394) took 1956 ms. Trace:
I20260709 12:17:15.478699 18373 rpcz_store.cc:276] 0709 12:17:13.252296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.283494 (+ 31198us) service_pool.cc:224] Handling call
0709 12:17:13.320385 (+ 36891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.431954 (+2111569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.444047 (+ 12093us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400116054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.444248 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4811788672}
I20260709 12:17:15.449168 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:15.484299 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1535) took 2103 ms. Trace:
I20260709 12:17:15.452621 20425 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.448508 20426 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:15.500486 18366 rpcz_store.cc:276] 0709 12:17:13.512880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.467770 (+1954890us) service_pool.cc:224] Handling call
0709 12:17:15.468225 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.468540 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.468566 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.468623 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:15.468654 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.468705 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.468805 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:15.501886 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.484701 18293 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:15.553889 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:15.535490 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 221670us
W20260709 12:17:15.563144 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.549288 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.495417 20383 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.551730 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:15.545261 19007 connection.cc:256] RPC call timeout handler was delayed by -1.0180747059998794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
I20260709 12:17:15.498827 20338 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:15.536335 18264 connection.cc:256] RPC call timeout handler was delayed by -1.9082267890000821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 12:17:15.580807 18444 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:15.582455 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:15.503778 20193 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:15.590608 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.590857 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.591301 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.591583 18293 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:15.591979 18296 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:15.592433 18373 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:15.592561 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.592866 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.593089 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1400) took 1934 ms. Trace:
I20260709 12:17:15.593182 18293 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:15.593679 20426 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.594199 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.593559 18373 rpcz_store.cc:276] 0709 12:17:13.658713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.508554 (+1849841us) service_pool.cc:224] Handling call
0709 12:17:15.592194 (+ 83640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.592851 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.593038 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.546873 20286 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.597170 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.602209 20383 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.603354 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 298) took 1811 ms. Trace:
I20260709 12:17:15.540457 20285 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:15.604079 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.604869 19041 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:15.491492 20427 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.498143 20153 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.550048 20356 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.593534 18296 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:15.590057 19036 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:15.500494 20127 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.179s
I20260709 12:17:15.604557 20285 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.506682 20134 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:15.589866 20020 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.614204 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:15.794817 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1451) took 2316 ms. Trace:
W20260709 12:17:15.682526 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.538938 18368 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:15.600512 18264 connection.cc:256] RPC call timeout handler was delayed by -1.747480319000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 12:17:15.621456 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.680814 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.603753 18367 rpcz_store.cc:276] 0709 12:17:13.791879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.575682 (+1783803us) service_pool.cc:224] Handling call
0709 12:17:15.602679 (+ 26997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.602947 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.602976 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.603032 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:15.603096 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.603151 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.603276 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.681244 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.606035 18297 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:17:15.711107 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:15.753347 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:15.962348 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.888674 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.882473 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.616888 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 299) took 1146 ms. Trace:
I20260709 12:17:15.846028 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:15.963560 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:15.962981 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 348843us
I20260709 12:17:15.615504 18299 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:15.896302 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:15.957578 18291 rpcz_store.cc:276] 0709 12:17:13.478587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.492027 (+ 13440us) service_pool.cc:224] Handling call
0709 12:17:15.297680 (+1805653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.347110 (+ 49430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.794514 (+447404us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54002364d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:15.794736 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4119137152}
I20260709 12:17:15.963554 18366 rpcz_store.cc:276] 0709 12:17:14.470371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.575867 (+1105496us) service_pool.cc:224] Handling call
0709 12:17:15.594618 (+ 18751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.616531 (+ 21913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.616559 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.616617 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:15.616661 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.616728 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:15.616836 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:15.841194 20032 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.658s	user 0.015s	sys 0.000s
W20260709 12:17:15.971475 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.967568 18449 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.968474 20134 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:15.968590 20239 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:15.968835 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.969099 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.589079 20207 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.969727 18440 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:15.969806 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:15.970106 20427 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.612797 20425 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.966984 19040 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:15.882368 18372 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:15.970443 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:15.971108 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.897917 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.615329 20408 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.900004 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.895458 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:15.978142 19038 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:17:15.982170 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 210371us
I20260709 12:17:15.582885 18370 rpcz_store.cc:276] 0709 12:17:13.379533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.417825 (+2038292us) service_pool.cc:224] Handling call
0709 12:17:15.455541 (+ 37716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.482424 (+ 26883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.482682 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.982918 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:15.982405 19044 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:17:16.013571 19043 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:15.970420 20020 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:15.640513 18293 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:15.613078 20286 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.023041 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.024000 18448 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:15.982481 18446 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:16.020331 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:15.605589 20160 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.778482 20431 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.590848 20193 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:15.976330 20134 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.983121 18444 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:15.980301 20359 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:15.977744 20225 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:16.026183 20383 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.026130 20390 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.026423 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.026908 20356 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.024344 20425 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.028848 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.028964 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:16.029021 20408 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.029850 20127 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:16.030887 18443 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:16.043628 20134 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.022701 20207 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.048991 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.049582 18443 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:16.050278 20207 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:16.050482 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.015699 20264 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.031566 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1396) took 2518 ms. Trace:
I20260709 12:17:16.051354 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.026084 20347 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.029132 19035 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:16.057211 18296 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:16.036231 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 300) took 1551 ms. Trace:
I20260709 12:17:16.056794 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.058075 19039 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.058943 18373 rpcz_store.cc:276] 0709 12:17:14.484491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.930836 (+1446345us) service_pool.cc:224] Handling call
0709 12:17:16.008534 (+ 77698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.017482 (+  8948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.017511 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:16.021393 (+  3882us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:16.035983 (+ 14590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.036053 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.036166 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:15.976701 20284 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:16.023869 18441 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:15.727355 20426 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:16.015936 18368 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1389) took 2698 ms (client timeout 1842 ms). Trace:
I20260709 12:17:16.046658 18299 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:16.028453 20020 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.044358 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.062127 18444 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:16.051575 18369 rpcz_store.cc:276] 0709 12:17:13.513442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.423849 (+1910407us) service_pool.cc:224] Handling call
0709 12:17:15.456797 (+ 32948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.457255 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.457285 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.457890 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.596037 (+138147us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:15.596073 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.596120 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.031319 (+435199us) raft_consensus.cc:1685] finished
0709 12:17:16.031343 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.031493 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6409,"spinlock_wait_cycles":34163072,"thread_start_us":137696,"threads_started":1,"wal-append.queue_time_us":524764}
I20260709 12:17:16.062979 18440 raft_consensus.cc:2410] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:16.051929 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.052194 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.053959 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.063273 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 301) took 1577 ms. Trace:
I20260709 12:17:16.066066 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.066948 18443 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:16.051862 20300 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.068444 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.064121 18447 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.068137 18449 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:17:16.068254 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.095778 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.096016 18296 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
W20260709 12:17:16.096475 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.097831 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.098122 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.098536 18440 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.065449 18367 rpcz_store.cc:276] 0709 12:17:14.485556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.966132 (+1480576us) service_pool.cc:224] Handling call
0709 12:17:16.019891 (+ 53759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.058634 (+ 38743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.058667 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:16.062966 (+  4299us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:16.063028 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.063086 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.063189 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:16.100054 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:16.087736 18371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.031s	sys 0.047s
W20260709 12:17:16.101364 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.053085 20127 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:16.102962 18371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.031s	sys 0.051s
I20260709 12:17:16.104481 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.106238 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.125947 20408 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.064998 20347 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.123695 18448 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:16.129269 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.130069 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:16.130167 18299 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:16.124233 18446 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:16.058413 20264 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.129882 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.132151 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.133296 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.134625 18445 raft_consensus.cc:2410] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:17:16.136270 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.136783 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.105429 20351 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.133258 18444 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:16.139420 18296 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:17:16.127969 20286 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:16.057227 20431 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.142786 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.146584 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.147483 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.105212 20284 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.150172 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.148357 18447 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.102561 20425 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.125319 20225 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:16.153201 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.158836 18443 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:16.159749 18448 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:16.157954 20032 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.155414 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.174294 18291 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.175906 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:16.061573 18368 rpcz_store.cc:269] 0709 12:17:13.281050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.321757 (+ 40707us) service_pool.cc:224] Handling call
0709 12:17:13.846653 (+524896us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b540019d320. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7fe18 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:15.280851 (+1434198us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b4c002f1980. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6711f644 00007f3c67146f2d 00007f3c67146b93 00007f3c68e7718b 00007f3c68e77cb5 00007f3c68e72f30 00007f3c68e747de 00007f3c68e8027c 00007f3c6de41b17
0709 12:17:15.977448 (+696597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176113408}
I20260709 12:17:16.179476 18447 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:16.172020 18375 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:16.182729 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.183488 18446 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.159315 20340 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.186928 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:16.161063 20153 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.208606 18445 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.212164 18299 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:16.142089 20351 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.216172 20020 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.216287 20386 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.217263 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.175881 20338 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.224771 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:16.227033 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.229382 20425 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.229453 20340 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.229882 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.254329 18449 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:16.232060 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:16.235430 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.236049 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1466) took 2300 ms. Trace:
I20260709 12:17:16.236428 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1399) took 2624 ms. Trace:
I20260709 12:17:16.255888 18444 raft_consensus.cc:2410] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:16.236977 18441 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:16.239936 20438 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:17:16.240749 20439 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:16.243673 18291 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:16.243819 18445 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:16.252645 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:16.262167 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 220385us
I20260709 12:17:16.171617 20127 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:16.256654 18371 rpcz_store.cc:276] 0709 12:17:13.611562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.433931 (+1822369us) service_pool.cc:224] Handling call
0709 12:17:15.434537 (+   606us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:15.452920 (+ 18383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.452972 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:15.470657 (+ 17685us) log.cc:844] Serialized 52 byte log entry
0709 12:17:16.132688 (+662031us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:16.132714 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.132756 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.235072 (+102316us) raft_consensus.cc:1685] finished
0709 12:17:16.235091 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.236361 (+  1270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2776,"raft.queue_time_us":17248,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":16533,"thread_start_us":654607,"threads_started":1,"wal-append.queue_time_us":710930}
I20260709 12:17:16.267092 18443 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:16.209364 20032 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.256521 18295 rpcz_store.cc:276] 0709 12:17:13.935410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.411766 (+1476356us) service_pool.cc:224] Handling call
0709 12:17:15.478190 (+ 66424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.507045 (+ 28855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.507076 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.554921 (+ 47845us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.995193 (+440272us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:15.995267 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.995390 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.235815 (+240425us) raft_consensus.cc:1685] finished
0709 12:17:16.235837 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.235988 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40537,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":238,"thread_start_us":187856,"threads_started":1,"wal-append.queue_time_us":591845}
I20260709 12:17:16.251466 20020 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:16.279402 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.280632 18448 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:16.283809 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.314989 18299 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:16.318225 18298 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
W20260709 12:17:16.321643 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.328979 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1461) took 2473 ms. Trace:
I20260709 12:17:16.287292 20020 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.333119 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.342082 20431 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.329515 18292 rpcz_store.cc:276] 0709 12:17:13.855814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.403247 (+1547433us) service_pool.cc:224] Handling call
0709 12:17:15.482960 (+ 79713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.542956 (+ 59996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.542989 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.543622 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.744261 (+200639us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:15.744317 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.744413 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.328729 (+584316us) raft_consensus.cc:1685] finished
0709 12:17:16.328757 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.328913 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2675,"raft.queue_time_us":38274,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"spinlock_wait_cycles":153344,"thread_start_us":200179,"threads_started":1,"wal-append.queue_time_us":588358}
I20260709 12:17:16.342931 20340 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.347501 18292 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:16.347393 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.348358 18367 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.348562 20032 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.367597 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:16.373430 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 405273us
I20260709 12:17:16.380620 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.381896 18295 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.385246 18446 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:16.276769 20338 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.388780 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:16.395303 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:16.395921 18447 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:16.322286 20193 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.397210 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.398321 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:16.400825 19039 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:16.400764 18296 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:17:16.401413 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.389304 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.403813 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.409781 18443 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:16.410748 18264 connection.cc:256] RPC call timeout handler was delayed by -1.6940267890001905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
I20260709 12:17:16.285004 20386 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.412611 18446 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:16.414520 18446 pending_rounds.cc:85] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
W20260709 12:17:16.414688 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93434.3us
W20260709 12:17:16.416221 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 391079us
I20260709 12:17:16.420704 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.397420 20367 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.410553 20153 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.414947 18446 pending_rounds.cc:107] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:16.425762 18446 raft_consensus.cc:2889] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:16.426354 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.431890 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.432276 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1397) took 2918 ms. Trace:
I20260709 12:17:16.432643 18374 rpcz_store.cc:276] 0709 12:17:13.513795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.447814 (+1934019us) service_pool.cc:224] Handling call
0709 12:17:15.500527 (+ 52713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.553399 (+ 52872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.553446 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.554045 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.713264 (+159219us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:15.713323 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.713420 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.432093 (+718673us) raft_consensus.cc:1685] finished
0709 12:17:16.432113 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.432225 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"thread_start_us":158737,"threads_started":1,"wal-append.queue_time_us":574737}
I20260709 12:17:16.434684 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.435357 18374 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:16.436332 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:16.436978 18374 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.402499 20431 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:16.444624 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:16.443763 18366 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:16.447067 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.467521 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:16.471210 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.422502 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:16.424137 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.473789 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:16.474750 18449 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:16.476080 18374 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:16.479993 20446 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:16.480566 20446 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:16.481009 20446 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.481874 20134 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.483569 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.485440 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.486960 20426 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:16.476578 20445 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:16.487313 20153 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.487938 20445 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:16.488467 20445 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:16.489626 20153 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.490236 20153 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.490921 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.492342 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:16.492939 20383 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:16.493461 20427 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.493672 20383 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:16.452565 20225 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.495162 20383 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.496407 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:16.497507 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:16.498071 20448 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:16.496481 20020 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.498713 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.504272 18297 raft_consensus.cc:2368] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:16.505678 20193 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:16.467223 20127 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.506366 20390 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:16.509174 18440 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:16.509749 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.452700 20284 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:16.512019 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.508394 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:16.511063 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:16.513140 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.520498 18371 raft_consensus.cc:2410] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:16.527033 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.523825 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.530419 18299 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:16.532835 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.511044 20193 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.514250 20300 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.537883 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.508126 18370 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:16.453506 20367 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.542680 20134 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.543188 20020 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.543416 20454 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.543833 20454 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:16.543864 20284 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:16.544093 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 57447.6us
I20260709 12:17:16.544826 20408 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:16.544899 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:16.545915 20408 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:16.546437 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.547016 20408 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.548503 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.550503 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.550930 20359 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:16.551113 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:16.552361 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.551857 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.555032 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.556545 18440 raft_consensus.cc:2410] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:16.558131 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1472) took 1054 ms. Trace:
I20260709 12:17:16.558492 18300 rpcz_store.cc:276] 0709 12:17:15.504168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.507330 (+  3162us) service_pool.cc:224] Handling call
0709 12:17:15.526929 (+ 19599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.612576 (+ 85647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.612605 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:15.612646 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:15.612683 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.557886 (+945203us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54002369d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:16.557935 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.558072 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169836288}
I20260709 12:17:16.562039 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 449) took 1223 ms. Trace:
I20260709 12:17:16.551415 20444 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:16.562649 20444 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.562417 19044 rpcz_store.cc:276] 0709 12:17:15.339159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.339440 (+   281us) service_pool.cc:224] Handling call
0709 12:17:15.339798 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:16.019282 (+679484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:16.561800 (+542518us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54008205d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:16.561973 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464271232}
I20260709 12:17:16.563375 20444 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.565697 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 678) took 1132 ms. Trace:
I20260709 12:17:16.537446 20300 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.566111 19036 rpcz_store.cc:276] 0709 12:17:15.433041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.450934 (+ 17893us) service_pool.cc:224] Handling call
0709 12:17:16.565482 (+1114548us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b540055e6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:16.565634 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19475328}
I20260709 12:17:16.559506 20359 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.567456 20351 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.567726 20359 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:16.536857 20193 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:16.569509 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.570343 18368 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:16.573405 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:16.573886 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.573696 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.574905 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.552277 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:16.529605 20153 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.571156 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.583648 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43588 (request call id 14) took 1269 ms. Trace:
W20260709 12:17:16.590389 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 94066us
I20260709 12:17:16.552161 20356 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.591459 18294 rpcz_store.cc:276] 0709 12:17:15.315733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.435322 (+119589us) service_pool.cc:224] Handling call
0709 12:17:16.548319 (+1112997us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540040db54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e82735 00007f3c6de42d29 00007f3c673c11d4 00007f3c673c1162 00007f3c673c1091 00007f3c673c100c 00007f3c673c02c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:16.554836 (+  6517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321018624}
I20260709 12:17:16.595546 18369 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:16.596107 18371 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:17:16.599316 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45623.6us
W20260709 12:17:16.608590 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.598575 18374 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.611828 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:16.618502 18374 raft_consensus.cc:2376] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4. Re-sending same reply.
I20260709 12:17:16.620337 18374 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:16.617434 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:16.622627 20020 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:16.622828 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.623337 20020 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:16.624456 20425 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.625072 20425 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:16.624785 20020 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.625639 20425 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.580410 20153 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:16.637058 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:16.701967 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65011.7us
I20260709 12:17:16.594997 20356 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.597594 20390 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:16.592393 20077 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.709441 20452 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.720455 20452 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:16.721459 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1916358529997524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 12:17:16.721874 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1671558139998979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
W20260709 12:17:16.722177 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1625706390000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24550000000000002s for a response
I20260709 12:17:16.722673 20452 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.703433 20134 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:17:16.713601 20456 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.714882 20444 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:16.716012 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 130382us
W20260709 12:17:16.719825 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet ea246fa8cc7e4edbb136da2ee92fb1d3: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:17:16.727519 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 179062us
I20260709 12:17:16.633538 18368 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:16.643052 20153 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.755475 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.699028 20338 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.708628 20356 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.707610 20225 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:16.761581 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 169130us
W20260709 12:17:16.762532 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 309358us
I20260709 12:17:16.765555 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.775899 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.778955 18368 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:16.705207 20446 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.768251 20408 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:16.766407 20225 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:16.827566 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:16.828065 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:16.828387 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.798497 18370 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:16.855611 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92008.8us
I20260709 12:17:16.823727 20408 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:16.855837 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 132919us
I20260709 12:17:16.807410 20284 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
W20260709 12:17:16.868103 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91516us
I20260709 12:17:16.822993 20225 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:16.864734 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 99270.4us
I20260709 12:17:16.821273 20462 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:16.787992 20456 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.768029 20454 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.865552 20408 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.863850 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:16.876791 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 112354us
I20260709 12:17:16.890285 20456 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.857198 20445 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.891793 20444 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.893715 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.895733 20284 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:16.896986 20264 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:16.923026 20207 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:16.924958 20461 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:17:16.927738 18338 connection.cc:256] RPC call timeout handler was delayed by -1.0612503349998406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:17:16.931571 20367 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:16.932785 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.934197 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.934803 18370 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
W20260709 12:17:16.935730 20460 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:17:16.937419 18338 leader_election.cc:400] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:16.937732 18338 leader_election.cc:403] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:16.939379 20459 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:16.940136 20265 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:16.885473 20468 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:17:16.897009 20446 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:16.923846 20457 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:17:16.942951 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 01de3fbc59f243ba8a35edaa5fcfe86f: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:16.940773 20265 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:16.887480 20462 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:16.924177 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.925081 20207 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.953588 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.953706 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.955459 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.955487 20462 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.956741 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.956987 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.970561 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:16.957444 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.971546 18295 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:16.958140 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.972633 18300 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:16.959831 18367 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:16.970742 18292 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:16.960692 18372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1536) took 3231 ms (client timeout 1521 ms). Trace:
I20260709 12:17:16.961748 20356 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.962649 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1481) took 1455 ms. Trace:
I20260709 12:17:16.975018 18297 rpcz_store.cc:276] 0709 12:17:15.506881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.028738 (+521857us) service_pool.cc:224] Handling call
0709 12:17:16.962369 (+933631us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540037f0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:16.962552 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007043968}
I20260709 12:17:16.962900 20408 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:16.973922 18372 rpcz_store.cc:269] 0709 12:17:13.729071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.858856 (+129785us) service_pool.cc:224] Handling call
0709 12:17:16.960484 (+3101628us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:16.960613 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4556260224}
I20260709 12:17:16.976876 18296 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 1.
I20260709 12:17:16.930198 20426 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:16.979760 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 109673us
I20260709 12:17:16.982740 18294 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:16.988511 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:16.989027 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:16.989485 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:16.990764 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:16.991040 20467 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.995991 20467 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:16.996439 20467 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.996484 20408 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:16.991824 20476 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.977799 20193 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:16.992115 20464 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:16.990768 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:17.002666 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 1d0efe89255942c4bd576ff4b96cd91d: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:17.003918 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:16.973253 20207 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.007296 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.008450 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.009120 20445 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.009224 20020 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:17.009536 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:17.009780 20020 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.010144 20020 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:16.982525 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54867us
I20260709 12:17:16.992678 20356 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.001760 20193 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:16.929375 20456 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:17:16.929608 20448 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:17:17.011161 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:17.033635 19990 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.034121 19990 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:17.034197 20153 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.034981 19990 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.026724 20020 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:16.940537 20425 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:17.012311 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.031119 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.050371 18369 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 2: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:17.039551 18412 connection.cc:256] RPC call timeout handler was delayed by -1.403285305999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:17:17.050992 18412 connection.cc:256] RPC call timeout handler was delayed by -1.3753853059999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260709 12:17:17.036896 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:17.051565 18412 connection.cc:256] RPC call timeout handler was delayed by -1.106285306000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22719999999999999s for a response
W20260709 12:17:17.031906 18375 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 292) took 3701 ms (client timeout 2547 ms). Trace:
W20260709 12:17:17.101011 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60088.8us
W20260709 12:17:17.101004 18375 rpcz_store.cc:269] 0709 12:17:13.330610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.333813 (+  3203us) service_pool.cc:224] Handling call
0709 12:17:17.031646 (+3697833us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b54000f9454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:17.031819 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6388914432}
I20260709 12:17:17.020789 20239 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.032087 20426 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.107769 20239 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.108249 20239 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.109575 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.109728 20426 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.110378 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:17.111200 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74378.1us
W20260709 12:17:17.111956 20300 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 12:17:17.112640 20338 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.112432 20300 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:17.120297 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.015146 20346 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:17.026806 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:17.119575 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.011417 20476 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:17.015503 20264 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.013036 20285 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:17.104408 20448 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:17.019599 18370 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:17.106001 20367 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:17.144060 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 219871us
I20260709 12:17:16.974013 20359 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:16.978636 20265 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.131917 20193 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.019522 20225 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.148026 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.147123 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:17.151203 18293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 310) took 1620 ms (client timeout 1816 ms). Trace:
I20260709 12:17:17.134694 20476 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:17.153105 18293 rpcz_store.cc:269] 0709 12:17:15.530407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.593938 (+ 63531us) service_pool.cc:224] Handling call
0709 12:17:17.151028 (+1557090us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b540004d654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:17.151151 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103048576}
I20260709 12:17:17.132256 20346 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:17.138063 20448 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:17.166733 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:17.152314 20193 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.152493 20472 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:17.165920 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.149994 20367 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.155344 20476 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.159224 20264 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.168875 20477 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:17.197301 20477 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.197750 20477 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:17.154739 20285 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:17:17.169777 20469 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:17:17.170688 20446 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.201148 18293 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:17.201406 20446 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.201886 20446 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.202358 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:17.199703 20472 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.204295 20472 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.205276 20225 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.203030 18293 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:17.211654 18291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 314) took 1612 ms (client timeout 1847 ms). Trace:
W20260709 12:17:17.212123 18291 rpcz_store.cc:269] 0709 12:17:15.601361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.972977 (+371616us) service_pool.cc:224] Handling call
0709 12:17:17.211483 (+1238506us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000aa0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:17.211594 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307970432}
I20260709 12:17:17.213485 18291 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:17.216028 20467 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.177748 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.218899 18370 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.184684 20265 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.219378 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:17.185439 19035 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.220098 18293 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.186029 20193 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.220664 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.220819 20479 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:17.222139 20479 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.222528 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.223276 20408 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.223693 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.186400 20346 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.219897 20477 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.224483 20356 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:17.191844 20264 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.225864 20408 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.185494 20367 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.281298 20386 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.217990 20467 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.301718 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.233866 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.261029 20359 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.300058 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.257220 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.231213 20454 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.270771 20193 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.241637 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.195117 20448 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:17.217960 19006 connection.cc:256] RPC call timeout handler was delayed by -1.0026513459999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 12:17:17.239559 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.227238 20264 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.256713 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.290202 20077 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:17.238137 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.279002 20346 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.288532 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:17.274523 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50868.8us
I20260709 12:17:17.302842 20347 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.245625 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.196228 20020 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.273916 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 305) took 1745 ms. Trace:
I20260709 12:17:17.192955 20153 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.272615 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.347023 18300 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:17.347980 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:17.417152 18292 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.314929 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:17.195868 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:17.348842 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:17.420006 20465 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.349664 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:17.420929 18294 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:17.420995 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 225234us
I20260709 12:17:17.384191 18370 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:17.422060 20465 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:17.295318 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:17.389950 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 277655us
I20260709 12:17:17.231830 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:17.424662 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 192896us
W20260709 12:17:17.425550 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:17.426134 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.404021 20431 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.426023 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.311924 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:17.429042 18297 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:17.169333 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.429910 18292 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:17.317672 20476 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.173686 20207 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.385095 20338 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:17.301105 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.400939 19043 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:17.431519 20467 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.316771 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:17.276995 20477 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:17.258922 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.435207 18294 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:17.406237 18367 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.440827 18374 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.218175 20446 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.443766 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.431439 19035 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:17.445351 20446 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.217478 20472 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:17.196079 20456 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.235s
I20260709 12:17:17.447479 20472 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:17.249500 20300 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:17.419126 18295 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:17.406663 20264 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.448789 20020 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.449998 20020 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.450495 20020 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.449072 20264 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.194370 20425 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.453116 20264 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.407774 20077 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:17.407734 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:17.430619 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 265139us
W20260709 12:17:17.404157 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 860404us
I20260709 12:17:17.588456 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:17.589061 18447 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.589988 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.591511 20153 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.591910 20153 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.592248 20153 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.595703 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.596712 19035 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.591120 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.600339 18367 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:17.454325 20077 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.457911 20193 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.461390 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.474483 18293 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:17.616899 20077 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:17.617159 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 155880us
I20260709 12:17:17.619050 20347 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.620469 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:17.621378 19037 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.604547 18373 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:17.402627 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.437918 20338 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.623263 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.412482 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:17.328984 20239 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.626897 20239 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:17.327791 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.634459 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.437964 20207 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.636059 20446 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:17.412477 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:17.636700 20446 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:17.636812 18371 raft_consensus.cc:606] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:17.637360 18371 raft_consensus.cc:614] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:17.411711 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:17.637656 20446 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.415196 18299 rpcz_store.cc:276] 0709 12:17:15.528431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.323919 (+795488us) service_pool.cc:224] Handling call
0709 12:17:16.324237 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:16.530802 (+206565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.273614 (+742812us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54002751d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:17.273843 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473717504}
I20260709 12:17:17.393055 20473 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.639444 20473 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.639770 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.641551 20473 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:17.413825 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:17.641742 20225 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:17.642278 18441 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.642452 20225 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:17.643513 20225 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.435418 20431 raft_consensus.cc:2764] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:17.456617 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.648988 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:17.649585 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 193030us
I20260709 12:17:17.650394 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:17.405707 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.651314 20207 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.651763 20207 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.572439 20448 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.513484 20020 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.574936 20476 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.653795 20476 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.654331 20476 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.654874 20476 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.408978 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.656627 18368 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:17.581605 20359 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.657523 20359 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.585310 18449 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:17.585688 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:17.660080 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.453640 19990 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:17.668874 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.600581 18447 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:17.602016 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.602609 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.674518 19040 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:17.602300 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:17.676086 19038 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.601238 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.604413 20425 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.677927 20425 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.678455 20425 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.605247 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:17.611616 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:17.681270 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93684.1us
I20260709 12:17:17.681994 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.682202 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:17.682720 20473 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.683899 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.684207 20346 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.684631 20473 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.684702 20346 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:17.685681 20346 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.686112 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:17.686487 20359 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:17.687036 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:17.687362 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 82199us
I20260709 12:17:17.688468 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.689463 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.690385 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.690943 19043 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 1.
I20260709 12:17:17.691340 18441 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:17.693754 20359 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.694361 20359 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.694581 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.694916 20359 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.696017 20020 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.696362 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:17.696470 20504 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:17.614046 19036 pending_rounds.cc:85] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 4
I20260709 12:17:17.702405 19036 pending_rounds.cc:107] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:17:17.702482 20504 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:17.702912 19036 raft_consensus.cc:2889] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:17.703708 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.615121 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.705628 19044 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.456748 18198 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.414791 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:17.624660 18375 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:17.624689 18443 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:17.623656 18369 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.706961 18442 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:17.706585 20504 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.627437 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:17.713474 20425 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.628607 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.709311 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.629354 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.717325 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.719089 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.707482 18198 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:17.723348 18198 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.726349 18368 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:17.631662 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.738215 19037 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:17.652223 20207 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.652851 18367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d4d1763576e4c49849611360f37665a"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:17.696703 20020 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.743719 20020 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.744172 20020 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.744390 20479 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:17.746197 18367 raft_consensus.cc:606] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:17.747108 18367 raft_consensus.cc:614] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:17.697919 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:17.698493 20431 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.699002 20473 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.749775 19040 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:17.699693 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.611394 20153 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:17.756438 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 56861.7us
I20260709 12:17:17.705536 19035 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:17.409516 18372 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
W20260709 12:17:17.762202 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61629.8us
I20260709 12:17:17.716150 18366 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:17.725189 18198 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.688467 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.737180 20284 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.756847 20077 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.767319 20448 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.749763 20431 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.768611 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.769042 20153 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:17.773382 19041 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:17.791114 20431 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.794412 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.795492 20452 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.795864 20452 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:17.796260 20452 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.796711 18373 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:17.778420 20077 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.799564 20077 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.799492 20300 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:17.799978 20465 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.800834 20300 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.801235 20300 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.801381 20465 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.801709 20300 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.802760 20465 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:17.803651 20153 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.804131 20153 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:17.805030 20153 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.807282 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:17.807672 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.808066 19990 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.803108 20465 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:17.808557 19990 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.808965 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.809250 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:17.810298 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:17.810060 20465 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.811431 20472 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:17.811403 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 122980us
I20260709 12:17:17.812150 20472 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.812201 20347 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.812659 20472 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.812748 20347 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:17.813313 20347 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.813817 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.777755 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:17.811424 20322 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:17.814704 20456 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.815124 20456 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:17.806457 20207 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.816485 18369 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:17.816427 18412 leader_election.cc:400] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:17.775090 18198 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.817991 18412 leader_election.cc:403] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:17.777212 20448 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.814232 20425 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:17.821965 18198 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 2 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:17.815521 20456 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.808982 19990 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:17.847733 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 130467us
I20260709 12:17:17.842242 18198 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:17.849366 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.850538 18198 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.830376 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:17.834703 20504 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:17.838095 18370 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:17.842866 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 139231us
I20260709 12:17:17.852569 18198 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.820397 20465 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:17.871352 20469 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:17.859478 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:17.864876 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 46062.3us
I20260709 12:17:17.823632 20285 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:17.865476 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47535.2us
I20260709 12:17:17.866912 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.825284 20454 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.779325 20284 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:17.883837 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.829604 20448 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.868614 18198 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.848320 20467 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.848822 20408 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:17.916595 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 93335.9us
I20260709 12:17:17.905449 18198 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 3, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 3 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:17.825467 20477 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:17:17.866582 20473 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.922650 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:17.924739 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:17.891019 20285 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.898649 20454 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.830065 20300 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:17.944304 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 77473us
I20260709 12:17:17.950920 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.951485 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.948374 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.948776 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:17.953310 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 104012us
W20260709 12:17:17.948099 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 56463.1us
I20260709 12:17:17.954061 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.955902 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:17.957254 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.867831 20504 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:17.957094 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.938196 20239 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:17.963732 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.970782 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.943167 20285 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.937949 20322 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:17.949683 20454 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.981094 18375 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:17.977087 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:17.952569 20225 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.975512 18371 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.979707 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:17.984092 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 117578us
I20260709 12:17:17.985502 18367 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:17.988211 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 157890us
I20260709 12:17:17.989038 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:17.958218 20473 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.971072 20511 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.950281 20386 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:18.003569 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.971017 20509 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.982360 20322 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.979768 20446 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:17.988063 20225 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.014966 20225 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:18.007259 18375 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:18.009423 18367 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:18.007917 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:18.013659 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:18.014241 20207 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:18.049525 18371 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:18.032188 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:18.130192 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.173633 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:18.022403 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.031683 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:18.036958 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.036362 20476 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.192677 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:17.984445 20506 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:18.206380 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 230809us
I20260709 12:17:18.029901 20431 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.187503 18374 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:18.143625 20508 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.133s	user 0.002s	sys 0.006s
W20260709 12:17:18.198372 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:18.215298 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.142439 18299 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:17.990051 20408 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:18.189523 18294 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:18.205765 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:18.216876 18371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:18.217415 18371 raft_consensus.cc:606] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:18.196676 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 183043us
W20260709 12:17:18.207096 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:18.129801 20498 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.145s	user 0.006s	sys 0.000s
I20260709 12:17:18.212083 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:18.137341 20497 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.009s
I20260709 12:17:18.028200 20284 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:18.010422 20471 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:18.011782 20509 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:18.220713 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 212880us
W20260709 12:17:18.226485 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 174909us
I20260709 12:17:18.171126 20193 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.020537 20300 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:18.189798 20512 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.236s	user 0.010s	sys 0.003s
I20260709 12:17:18.114521 20477 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:18.187001 20322 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:18.203887 20207 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:18.018998 20347 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.036237 20446 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:18.006249 20520 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:18.228004 18442 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.227602 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:18.024384 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:18.233964 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:18.035905 20511 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:18.237551 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:18.019181 20020 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.243911 18296 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.029556 20077 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.221659 20408 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:18.235307 20359 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
I20260709 12:17:18.309845 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.311553 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.230165 20509 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.225732 20469 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:18.193878 20225 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.033883 20452 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.194849 20448 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.307088 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:18.223500 20346 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:18.303426 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58100.6us
I20260709 12:17:18.243623 20471 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:18.238754 20207 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.238330 20477 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:18.361459 18375 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:18.365878 18374 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:18.035627 20386 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.307670 20465 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:18.370828 18299 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 5. Ops: []
I20260709 12:17:18.357191 20469 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:18.367200 20520 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:18.369060 20468 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:18.358456 20359 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.379524 18447 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:18.398815 18373 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:18.237388 20193 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.375546 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.240506 20517 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.147s
I20260709 12:17:18.407693 18292 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.434576 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.571599 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.597512 18294 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:18.597817 18440 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:18.556756 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:18.433341 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.599859 18449 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:18.549990 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:18.605141 18296 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:18.438982 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.607501 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.598654 18291 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:18.440481 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.435799 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.610026 18293 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:18.437685 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.433930 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.612233 18445 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:18.572634 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.003s	sys 0.007s
W20260709 12:17:18.522987 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 295714us
W20260709 12:17:18.375730 20507 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:17:18.375772 20477 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:18.614073 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.004s	sys 0.008s
I20260709 12:17:18.619627 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:18.647244 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.406864 20346 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.683604 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:18.433591 20499 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 12:17:18.685456 18372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.011s	sys 0.004s
W20260709 12:17:18.686084 18372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.011s	sys 0.004s
I20260709 12:17:18.379093 20527 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.675745 18440 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.375662 20207 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:18.681267 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:18.662603 18339 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.005s	sys 0.010s
W20260709 12:17:18.701244 18339 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.005s	sys 0.010s
W20260709 12:17:18.701622 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 617886us
I20260709 12:17:18.600291 20469 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:18.435925 20476 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:18.689081 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69539.9us
I20260709 12:17:18.703866 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:18.691797 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.613451 20239 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:18.674333 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 790549us
I20260709 12:17:18.406490 20504 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:18.367516 20523 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.334s
I20260709 12:17:18.703758 18441 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.379899 20322 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.705112 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:18.704337 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.705735 18293 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:18.701386 18300 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:18.523098 20454 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.712286 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.375187 20456 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.718008 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:18.717789 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:18.704387 20431 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:18.712692 18370 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:17:18.727835 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.728845 18293 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:18.690238 18291 raft_consensus.cc:2410] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:18.716566 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:18.692756 20473 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:18.717325 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:18.727396 18369 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:18.715794 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.714499 20479 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:18.435051 20446 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.435926 20471 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.731768 18444 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.732362 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:18.738111 18441 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:18.738842 18294 raft_consensus.cc:2410] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:18.722152 20531 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:18.736178 18291 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.742561 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.744182 18293 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:18.714407 20239 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:18.747179 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:18.763190 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:18.730548 20473 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:18.703763 20527 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:18.617446 20511 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.732578 20479 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:18.791642 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:18.791862 18441 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:18.793926 19042 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:18.741043 20347 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:18.743785 20284 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:18.790299 20531 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:18.810967 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:18.824745 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 106799us
I20260709 12:17:18.755963 20468 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:18.831171 18296 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:18.708449 20520 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:18.794605 20472 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.981s	user 0.013s	sys 0.000s
I20260709 12:17:18.793219 20448 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:18.799736 20077 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:18.804044 20479 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:18.794267 20540 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:18.907702 20540 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:18.908144 20540 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:18.909134 20533 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:17:18.910671 20468 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.829770 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 694) took 1215 ms. Trace:
I20260709 12:17:18.911659 20347 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:18.800720 20465 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:18.812579 20020 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:18.941129 20322 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:18.959887 20454 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:18.959749 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:18.961722 18297 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
W20260709 12:17:18.912689 20522 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 12:17:18.912994 20536 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:17:18.978729 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1425) took 1025 ms. Trace:
W20260709 12:17:18.979576 18412 connection.cc:256] RPC call timeout handler was delayed by -1.7089011030000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 12:17:18.980059 18412 connection.cc:256] RPC call timeout handler was delayed by -1.6651111700000456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
I20260709 12:17:18.979115 18372 rpcz_store.cc:276] 0709 12:17:17.953102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.953243 (+   141us) service_pool.cc:224] Handling call
0709 12:17:17.973971 (+ 20728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:17.974326 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.974354 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:17.985410 (+ 11056us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.686576 (+701166us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.686625 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.686704 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:18.978528 (+291824us) raft_consensus.cc:1685] finished
0709 12:17:18.978548 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.978675 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":481,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9146,"thread_start_us":571475,"threads_started":1,"wal-append.queue_time_us":719397}
W20260709 12:17:18.863834 20285 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.872s	user 0.009s	sys 0.001s
I20260709 12:17:18.980751 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:18.941852 20525 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 12:17:18.943522 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 231248us
W20260709 12:17:18.968089 19990 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.077s	user 0.005s	sys 0.005s
W20260709 12:17:18.927932 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.004705 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 208113us
I20260709 12:17:19.006578 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:18.952565 20239 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:19.007366 18297 raft_consensus.cc:2376] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2. Re-sending same reply.
I20260709 12:17:18.913864 20386 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:19.008308 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:19.007411 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.003366 20285 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.016090 20285 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:18.859338 20472 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:18.969132 20359 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:19.011432 19990 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.014161 20527 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:18.941561 20284 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:17:19.036172 18442 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:19.037204 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:19.006011 20454 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.009738 20346 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:19.046398 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.049041 20465 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.035257 20239 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:17:18.982573 20473 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.050549 18299 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:19.273607 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1430) took 1311 ms. Trace:
I20260709 12:17:19.050717 18297 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:19.198549 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.053702 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.324193 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1892) took 1181 ms. Trace:
I20260709 12:17:19.058694 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.649492 18293 raft_consensus.cc:2376] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3. Re-sending same reply.
I20260709 12:17:19.053117 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1436) took 1019 ms. Trace:
W20260709 12:17:19.320278 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 1.00077e+06us
I20260709 12:17:19.447988 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:19.326090 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1424) took 1373 ms. Trace:
I20260709 12:17:19.665326 18369 rpcz_store.cc:276] 0709 12:17:17.952786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.958912 (+  6126us) service_pool.cc:224] Handling call
0709 12:17:19.325808 (+1366896us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400414454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:19.325986 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710354816}
I20260709 12:17:19.061224 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.662974 18374 rpcz_store.cc:276] 0709 12:17:18.033504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.380234 (+346730us) service_pool.cc:224] Handling call
0709 12:17:18.380569 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.380919 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.380952 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.613437 (+232485us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.803024 (+189587us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.803074 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.803141 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:19.052823 (+249682us) raft_consensus.cc:1685] finished
0709 12:17:19.052858 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.053041 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18662,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":34211,"thread_start_us":87993,"threads_started":1,"wal-append.queue_time_us":364919}
I20260709 12:17:19.309489 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:19.271937 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 263662us
I20260709 12:17:19.057453 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:19.239161 20207 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 12:17:19.461702 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:19.299669 18299 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.911856 19036 rpcz_store.cc:276] 0709 12:17:17.613301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.613426 (+   125us) service_pool.cc:224] Handling call
0709 12:17:17.613808 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:17.703698 (+ 89890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.703741 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:17.704292 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:17:17.833875 (+129583us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:17.833940 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.834061 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:18.744967 (+910906us) raft_consensus.cc:1685] finished
0709 12:17:18.744994 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.828479 (+ 83485us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129130,"threads_started":1,"wal-append.queue_time_us":431699}
W20260709 12:17:19.601887 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.274063 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 227759us
I20260709 12:17:19.669137 20553 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:19.593879 20509 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.187s	user 0.012s	sys 0.000s
W20260709 12:17:19.670791 20556 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:17:19.672741 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.638164 20452 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:19.064608 20472 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:19.054119 20454 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.674260 20239 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.017959 20322 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:19.054966 20284 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.306231 20514 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.674906 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1959) took 1430 ms. Trace:
I20260709 12:17:19.649833 18292 rpcz_store.cc:276] 0709 12:17:18.142958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.143193 (+   235us) service_pool.cc:224] Handling call
0709 12:17:18.181766 (+ 38573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.596522 (+414756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.323875 (+727353us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540040cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:19.324114 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518966528}
I20260709 12:17:19.666968 18297 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:19.740129 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1916) took 1031 ms. Trace:
I20260709 12:17:19.740592 18297 rpcz_store.cc:276] 0709 12:17:18.708471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.296029 (+587558us) service_pool.cc:224] Handling call
0709 12:17:19.649201 (+353172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.739812 (+ 90611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.740062 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:19.650305 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.274425 18375 rpcz_store.cc:276] 0709 12:17:17.962544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.371778 (+409234us) service_pool.cc:224] Handling call
0709 12:17:18.414367 (+ 42589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.414637 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:18.560221 (+145584us) log.cc:844] Serialized 14 byte log entry
0709 12:17:19.064004 (+503783us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b180045a900. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:17:19.273215 (+209211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.273269 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:19.273324 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.273393 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.273548 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142261,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":97,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":36209,"spinlock_wait_cycles":33262848,"thread_start_us":208914,"threads_started":1,"wal-append.queue_time_us":871019}
I20260709 12:17:19.676836 18296 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:19.273031 20467 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.351s	user 0.013s	sys 0.000s
W20260709 12:17:19.683631 20446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.007s	sys 0.007s
I20260709 12:17:19.676846 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1912) took 1079 ms. Trace:
W20260709 12:17:19.665993 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3264759159999358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:17:19.669447 20521 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:19.450120 20408 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.082s	user 0.007s	sys 0.003s
W20260709 12:17:19.474118 20540 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
W20260709 12:17:19.490423 20469 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.767s	user 0.010s	sys 0.000s
I20260709 12:17:19.150457 20448 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:19.005903 20517 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.699620 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:19.700361 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:19.495678 20479 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 12:17:19.017446 20537 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:19.424883 20473 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:19.720890 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.726404 19044 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:19.721244 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.756678 18445 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
W20260709 12:17:19.562085 20506 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.327s	user 0.012s	sys 0.000s
I20260709 12:17:19.720525 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.599754 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.732123 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.748952 20467 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:19.759850 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:19.506114 20477 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.001s
I20260709 12:17:19.601683 20193 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:19.066843 20456 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:19.067014 20359 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.066043 19990 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:19.770102 18440 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:19.770295 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:19.273908 20386 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.737977 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:19.742046 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.738770 18444 rpcz_store.cc:276] 0709 12:17:18.244834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.244957 (+   123us) service_pool.cc:224] Handling call
0709 12:17:18.356701 (+111744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.738073 (+381372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.674637 (+936564us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400185bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:19.674813 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828495616}
I20260709 12:17:19.677228 19043 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:19.746579 18299 rpcz_store.cc:276] 0709 12:17:18.597135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.298700 (+701565us) service_pool.cc:224] Handling call
0709 12:17:19.299314 (+   614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.676562 (+377248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.676755 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:19.747063 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2754157520000717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 12:17:19.054780 20546 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.685s
I20260709 12:17:19.781713 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:19.781885 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2335693189997983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 12:17:19.781771 20546 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:19.782460 20546 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:19.782702 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2214157519999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 12:17:19.782990 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.759541 20506 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.783923 20506 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:19.697714 20285 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 12:17:19.746472 20446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.008s	sys 0.008s
I20260709 12:17:19.739010 20284 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:19.736686 20239 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:19.753291 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.751344 20521 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.754622 18296 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.794615 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:19.754334 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.738621 20514 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:19.795609 18447 raft_consensus.cc:2368] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:19.749547 20225 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:19.722851 20338 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:17:19.798166 18295 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 5. Ops: []
I20260709 12:17:19.795709 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.754534 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:19.196168 20528 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.698166 20207 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.731736 20509 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.771001 18369 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:19.757726 20465 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:19.062191 20531 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.773309 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:19.783080 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1602157520001128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
I20260709 12:17:19.826259 18298 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:19.826152 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.784466 18294 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:19.826615 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1450443720000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
I20260709 12:17:19.797958 20446 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.786175 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:19.758013 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.756659 20527 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:19.750669 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:19.756202 20448 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.787906 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:19.788621 19037 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:19.721315 20553 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.790318 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.751854 20408 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.767870 20347 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.761022 20322 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:19.793155 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.753032 18293 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:19.752310 20540 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.752477 20469 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.815379 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:19.815653 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61757.5us
I20260709 12:17:19.772472 20477 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.816318 18372 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.872530 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.816787 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:19.814564 20514 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.817394 20338 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:19.817798 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:19.819614 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.758512 20479 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.820587 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:19.823716 19044 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:19.881299 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 474) took 1125 ms. Trace:
I20260709 12:17:19.881843 19044 rpcz_store.cc:276] 0709 12:17:18.756418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.756613 (+   195us) service_pool.cc:224] Handling call
0709 12:17:19.672597 (+915984us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:19.881239 (+208642us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4708608}
I20260709 12:17:19.882467 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:19.761013 20517 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.794524 20551 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:19.824544 18367 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:19.783568 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:19.826946 18338 connection.cc:256] RPC call timeout handler was delayed by -1.0933759159997862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 12:17:19.749969 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.885902 18446 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:19.831140 18443 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:19.886165 18292 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.876093 20338 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:19.816810 20560 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:19.844544 20285 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.828231 18370 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:19.865960 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:19.869908 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:19.871404 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.871608 18374 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.873220 19041 raft_consensus.cc:2368] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:19.873474 18440 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:19.791316 20506 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.874158 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.877727 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:19.822964 20528 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.883319 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.883981 19039 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.884217 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.867924 20347 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.829000 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 1972) took 1127 ms. Trace:
I20260709 12:17:19.925237 18448 rpcz_store.cc:276] 0709 12:17:18.697240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.699842 (+  2602us) service_pool.cc:224] Handling call
0709 12:17:19.824230 (+1124388us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400258ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:19.824388 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349384576}
I20260709 12:17:19.828848 19042 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:19.839795 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:19.887735 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:19.881358 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.832340 20465 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.895018 18292 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:19.896049 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:19.928992 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.929333 18372 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:19.832621 20446 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.900805 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:19.879443 20454 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:19.842301 20527 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.829608 20284 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.815210 20546 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:19.885952 18338 connection.cc:256] RPC call timeout handler was delayed by -1.0055443720000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
I20260709 12:17:19.923868 20518 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:19.885038 20456 raft_consensus.cc:2764] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:19.927171 18447 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:19.927505 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:19.927765 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:19.869732 20322 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.926342 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 613) took 1019 ms. Trace:
I20260709 12:17:19.965602 19044 rpcz_store.cc:276] 0709 12:17:18.907317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.883016 (+975699us) service_pool.cc:224] Handling call
0709 12:17:19.926278 (+ 43262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:19.886858 20468 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:19.930855 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.897997 20514 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.931353 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:19.931727 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.932818 18445 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:19.909907 19035 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:19.864130 20239 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:19.958724 18296 raft_consensus.cc:2368] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:19.932894 20557 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:19.960430 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.908897 20560 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:19.909832 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 611) took 1003 ms. Trace:
I20260709 12:17:19.923755 20472 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:19.888532 20467 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:17:19.924235 19990 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:19.966779 18297 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:19.983734 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61511.1us
I20260709 12:17:19.968426 18292 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:19.959390 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 606) took 1108 ms. Trace:
I20260709 12:17:19.972064 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.978843 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.977670 19041 rpcz_store.cc:276] 0709 12:17:18.906684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.872351 (+965667us) service_pool.cc:224] Handling call
0709 12:17:19.909758 (+ 37407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:19.979840 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:19.981542 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.982566 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.986377 18447 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:19.986572 18375 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.987022 18445 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:19.909026 20285 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.984583 19043 rpcz_store.cc:276] 0709 12:17:18.851144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.884573 (+1033429us) service_pool.cc:224] Handling call
0709 12:17:19.959328 (+ 74755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:19.896673 20551 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.994585 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.995983 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.988392 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.988839 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:19.990525 20472 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:19.990658 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:19.996064 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:19.996878 19044 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:19.976356 20557 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:19.976905 20452 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:20.030781 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.001955 20547 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.981947 20515 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:20.002688 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.002740 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.002993 20551 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.004383 20472 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:19.989724 20509 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.005470 20563 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:20.006397 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.077337 18447 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:20.076220 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 104214us
I20260709 12:17:20.078199 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.078891 18447 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:20.079622 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.080134 18447 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:20.080803 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:20.009758 18298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1905) took 1634 ms (client timeout 1666 ms). Trace:
W20260709 12:17:20.081398 18298 rpcz_store.cc:269] 0709 12:17:18.375096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.406149 (+ 31053us) service_pool.cc:224] Handling call
0709 12:17:20.009567 (+1603418us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400002154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.009699 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804101888}
I20260709 12:17:20.010265 18300 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.082356 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.992146 20193 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:20.083287 20193 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:20.084249 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:19.993811 20207 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:20.086434 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.085582 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.012956 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.013397 20446 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.014524 20347 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.015355 20540 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.088397 20547 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:20.016109 20479 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:20.123819 18298 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:20.124042 20547 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.124830 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:20.125500 18298 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:20.126362 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.049445 20515 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.127379 20535 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:20.127565 20515 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:19.992496 20566 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:17:20.128047 20446 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.128643 20566 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.128737 20446 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.129067 20566 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:20.129463 20446 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:20.129643 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.129874 20446 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:20.130019 20566 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.017776 20528 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.018591 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1888) took 1988 ms. Trace:
I20260709 12:17:20.131976 18295 rpcz_store.cc:276] 0709 12:17:18.030274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.031356 (+  1082us) service_pool.cc:224] Handling call
0709 12:17:18.031790 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.805624 (+1773834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.018296 (+212672us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400001754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.018512 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4061879424}
I20260709 12:17:20.133301 18295 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.975952 20468 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.019968 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.135469 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.006990 20511 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:20.021386 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.021468 19990 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.023399 20527 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:20.024163 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.024103 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.024861 20322 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.024974 20448 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.026084 20506 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.027356 20517 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:19.989125 20562 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 12:17:19.989333 20561 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:17:19.982263 20546 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.071081 20452 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:19.983379 20239 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:20.136314 20571 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:17:20.136628 20408 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:20.002015 20285 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.050979 18445 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:20.005316 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:20.075479 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 152938us
I20260709 12:17:20.084115 20193 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.012128 20553 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.127698 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.089255 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.136739 18297 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.139508 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1902) took 1765 ms. Trace:
I20260709 12:17:20.152473 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:20.048556 20557 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.145509 19043 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.147529 18442 raft_consensus.cc:2368] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:20.147636 19040 raft_consensus.cc:2368] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
W20260709 12:17:20.148361 19037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 471) took 1397 ms (client timeout 1815 ms). Trace:
W20260709 12:17:20.073566 20558 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
I20260709 12:17:19.968502 20338 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.143741 18264 leader_election.cc:400] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:20.028455 20521 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:20.180696 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 206037us
I20260709 12:17:20.166070 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:20.167281 18294 rpcz_store.cc:276] 0709 12:17:18.374284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.740729 (+366445us) service_pool.cc:224] Handling call
0709 12:17:18.784268 (+ 43539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.826881 (+1042613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.139176 (+312295us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b540004c4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.139436 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299775872}
I20260709 12:17:20.172812 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.228736 18264 leader_election.cc:403] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:20.156318 18292 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:20.173115 18298 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:20.173727 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:20.186848 19037 rpcz_store.cc:269] 0709 12:17:18.751149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.761585 (+ 10436us) service_pool.cc:224] Handling call
0709 12:17:19.599524 (+837939us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:20.148157 (+548633us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400748bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.148301 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99928704}
I20260709 12:17:20.173797 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.232116 18448 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:20.155423 20552 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.661s	user 0.007s	sys 0.000s
I20260709 12:17:20.233882 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:20.200913 18415 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 2.094s (SENT) [suppressed 37 similar messages]
W20260709 12:17:20.234436 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 477213us
I20260709 12:17:20.238638 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.240672 19044 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:20.077490 20523 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:20.246989 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.248968 18292 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:20.250048 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 84100.2us
I20260709 12:17:20.154886 19990 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.161746 20465 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.398712 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1906) took 2023 ms. Trace:
I20260709 12:17:20.174117 20506 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.299635 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.301941 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.300364 18294 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:20.309360 20528 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:20.248868 20468 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.319855 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.308712 20580 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.436236 18292 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.343065 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.314184 18445 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.339057 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 604) took 1510 ms. Trace:
I20260709 12:17:20.364372 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.403584 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:20.344837 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.250743 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 150612us
W20260709 12:17:20.398938 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 198046us
I20260709 12:17:20.393803 18447 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:20.308506 18263 leader_election.cc:400] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
W20260709 12:17:20.439725 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 131347us
W20260709 12:17:20.440047 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.156046 20579 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:20.334290 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:20.445456 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 111244us
I20260709 12:17:20.446460 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.448089 20585 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:20.449091 20585 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:20.412778 20473 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.653s	user 0.005s	sys 0.006s
I20260709 12:17:20.308631 20576 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:20.323252 20338 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.329665 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.346871 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.407991 18291 rpcz_store.cc:276] 0709 12:17:18.381776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.789129 (+407353us) service_pool.cc:224] Handling call
0709 12:17:18.789588 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.789884 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.789922 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.801226 (+ 11304us) log.cc:844] Serialized 26 byte log entry
0709 12:17:19.270819 (+469593us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:19.270875 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.270986 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:20.078168 (+807182us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:20.394321 (+316153us) raft_consensus.cc:1685] finished
0709 12:17:20.394343 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.398621 (+  4278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11259776,"thread_start_us":327866,"threads_started":1,"wal-append.queue_time_us":595623}
I20260709 12:17:20.452483 18447 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:20.202023 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.453919 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.454324 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.419625 18373 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:20.454630 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51142.1us
W20260709 12:17:20.454663 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.456940 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:20.419795 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 117919us
I20260709 12:17:20.305418 18371 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:20.342087 18375 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:20.458004 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.156299 20477 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:20.427209 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.459014 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:20.251464 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.460582 18366 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:20.441329 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:20.437528 19039 rpcz_store.cc:276] 0709 12:17:18.850487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.905901 (+ 55414us) service_pool.cc:224] Handling call
0709 12:17:19.828057 (+922156us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:19.828239 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.923706 (+ 95467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.338726 (+415020us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540048f8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.338954 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132385408}
I20260709 12:17:20.302057 20527 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260709 12:17:20.218176 20347 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:20.335603 20193 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.461602 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:20.435542 20408 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.165217 20535 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.408242 20452 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.302225 20511 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 12:17:20.343501 20322 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 12:17:20.461881 20576 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.465586 18447 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:20.466290 18440 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:20.420257 20528 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.467308 18294 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:20.504809 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:20.457320 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.456060 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:20.476094 18374 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:20.313032 20446 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.483667 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.484513 18445 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.479111 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:20.479681 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.427469 20523 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.509341 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.233527 20551 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.232s
I20260709 12:17:20.490365 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.491258 18375 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:20.484434 20477 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.462806 20465 raft_consensus.cc:2764] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:20.466887 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.509301 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:20.435653 20549 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:20.510079 18298 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.483641 20564 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.508352 20528 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:20.521839 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 75470.5us
I20260709 12:17:20.512542 19040 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.301115 20553 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.509217 20467 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.499388 20509 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.516647 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:20.504901 20576 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.462059 20579 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.520009 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.476989 20473 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:20.521476 18445 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:20.521605 19037 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 1.
I20260709 12:17:20.521883 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.511495 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:20.523976 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:20.477726 20347 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.546985 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.324008 20560 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.477391 20574 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:17:20.507848 20408 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 12:17:20.564368 20589 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:17:20.564484 20520 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.564960 20520 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:20.514159 19990 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:17:20.566705 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.567771 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:20.573529 18291 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:20.524395 18368 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:20.576287 20583 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
W20260709 12:17:20.576932 20575 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:17:20.576705 20555 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
W20260709 12:17:20.577814 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 103264us
I20260709 12:17:20.517052 20477 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.548187 20467 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.554562 20465 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.526541 20322 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:17:20.617795 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1969) took 2095 ms. Trace:
I20260709 12:17:20.595839 20566 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.596213 20594 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:20.522526 20549 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.600455 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:20.647791 18296 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:20.647660 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 124367us
I20260709 12:17:20.608122 19990 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:20.548290 20553 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.637128 18367 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.638482 20520 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.642410 20540 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.643720 20322 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:20.667357 20477 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:20.667651 20514 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.000s
I20260709 12:17:20.643821 20578 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:20.668648 20578 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:20.669495 20578 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.670666 20578 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:20.620643 18446 rpcz_store.cc:276] 0709 12:17:18.547627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.555284 (+  7657us) service_pool.cc:224] Handling call
0709 12:17:18.555649 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.886470 (+1330821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.617512 (+731042us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003561d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.617721 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3058841344}
I20260709 12:17:20.671617 20537 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:20.672043 20537 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.672919 20579 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:20.651665 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 131693us
I20260709 12:17:20.653689 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:20.654470 18374 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:20.675734 18297 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.655853 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:20.636474 20386 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.570262 20473 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:20.662411 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:20.665254 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 199303us
I20260709 12:17:20.659746 19990 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:20.666724 20520 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.681735 20386 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:20.684002 20386 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.685075 20473 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.667058 20540 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:20.688596 20540 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.690049 20540 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.667511 20322 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:20.675271 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.651127 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.691931 20322 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.692883 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.693068 20551 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.693620 20579 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.694131 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.678694 19040 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:20.695459 20551 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:20.695180 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:20.680222 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:20.697618 19044 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.697804 20386 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:20.698165 20386 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:20.610209 20577 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:20.681550 19990 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.698691 20577 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:20.701164 20515 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.002s
I20260709 12:17:20.701582 20386 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.701646 20515 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.702972 19042 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:20.698213 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 617) took 1789 ms. Trace:
I20260709 12:17:20.703891 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 608) took 1845 ms. Trace:
I20260709 12:17:20.703835 19044 rpcz_store.cc:276] 0709 12:17:18.908460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.697184 (+1788724us) service_pool.cc:224] Handling call
0709 12:17:20.697450 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.697979 (+   529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.698164 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.704267 19042 rpcz_store.cc:276] 0709 12:17:18.858432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.951415 (+1092983us) service_pool.cc:224] Handling call
0709 12:17:20.703726 (+752311us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54004777d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.703842 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1670092032}
I20260709 12:17:20.705448 19042 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:20.705603 20239 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.002s
W20260709 12:17:20.706887 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.708416 18446 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:20.709509 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.710053 18446 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:20.710898 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.712016 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.715193 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:20.718133 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.712424 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.718811 20518 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:20.719184 20518 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.719504 18445 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.719694 20454 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:20.720005 20574 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.712653 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.715946 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.713068 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:20.713253 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.708287 20553 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.700793 20577 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:20.745877 18297 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:20.743906 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.721469 18440 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:20.676764 19043 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
W20260709 12:17:20.724682 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 206292us
I20260709 12:17:20.693004 20322 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:20.737565 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 62411.5us
I20260709 12:17:20.748245 19041 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.697638 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:20.671078 20469 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:20.749511 19043 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.705268 19044 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:20.751840 18445 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:20.525282 20452 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.735670 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:20.752206 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:20.719522 20518 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:20.784485 20239 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:20.785204 20535 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:20.786114 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50543.9us
I20260709 12:17:20.786154 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.786527 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:20.716094 20477 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.793243 19043 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:20.796994 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 99401.5us
I20260709 12:17:20.797760 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.798086 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.798202 20601 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:20.798666 20601 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:20.799037 20601 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.799144 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.791435 20239 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:20.800833 20239 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.803833 19036 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:20.805406 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.809664 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.810186 18366 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:20.751705 20465 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:20.813959 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.817253 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.817900 18441 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.816030 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:20.806128 20577 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.819680 18448 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:20.809494 20553 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:20.794301 20551 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:17:20.826398 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:20.827214 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.819787 20602 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:20.811569 20467 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:20.833446 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.836771 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:20.823697 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.825973 20553 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.847711 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:20.754683 20574 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:17:20.804706 20599 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:17:20.835938 20579 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.853533 18368 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5.
I20260709 12:17:20.855773 19990 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.856608 20523 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:20.863308 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:20.843709 20467 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:20.864733 18371 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.865675 18371 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:20.825579 20408 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:17:20.866430 18371 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.849761 20553 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:20.857884 20239 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:20.881058 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.882202 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:20.885869 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1962) took 2570 ms. Trace:
W20260709 12:17:20.872929 20546 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.005s
I20260709 12:17:20.886242 18443 rpcz_store.cc:276] 0709 12:17:18.311948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.358750 (+ 46802us) service_pool.cc:224] Handling call
0709 12:17:18.369784 (+ 11034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.886556 (+1516772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.881735 (+995179us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003b24d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.881897 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3356965120}
I20260709 12:17:20.887338 19009 leader_election.cc:400] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5.
I20260709 12:17:20.887753 19009 leader_election.cc:403] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:20.867296 20467 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:20.890082 18371 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.890146 20467 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.866386 20347 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:20.887687 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.901696 18369 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:20.875029 20452 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:20.917248 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.918689 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.926493 18443 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:20.886142 20602 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.927522 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:20.900593 20514 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5.
I20260709 12:17:20.918349 20452 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:20.943922 20600 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:17:20.927295 20347 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.980970 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 709) took 1135 ms. Trace:
I20260709 12:17:20.981460 19042 rpcz_store.cc:276] 0709 12:17:19.845466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.706404 (+860938us) service_pool.cc:224] Handling call
0709 12:17:20.979824 (+273420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.979998 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:20.980022 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:20.980233 (+   211us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:20.980278 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.980336 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.980822 (+   486us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400640ad0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.980918 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628144384}
W20260709 12:17:20.982539 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.927778 20546 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 12:17:20.988713 19036 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 616) took 2080 ms (client timeout 1578 ms). Trace:
W20260709 12:17:20.989203 19036 rpcz_store.cc:269] 0709 12:17:18.908143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.735942 (+827799us) service_pool.cc:224] Handling call
0709 12:17:20.988518 (+1252576us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b54006202d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.988633 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938441472}
I20260709 12:17:20.991592 20446 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.947873 20602 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:20.993111 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1542) took 1113 ms. Trace:
I20260709 12:17:20.993605 18449 rpcz_store.cc:276] 0709 12:17:19.879964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.519092 (+639128us) service_pool.cc:224] Handling call
0709 12:17:20.609317 (+ 90225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.621830 (+ 12513us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:20.621873 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:20.621928 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:20.621971 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.992353 (+370382us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.992398 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.992949 (+   551us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54001860d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:20.993058 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30093,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":14108,"spinlock_wait_cycles":162806272}
I20260709 12:17:20.991693 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1900) took 2618 ms. Trace:
I20260709 12:17:20.995492 18300 rpcz_store.cc:276] 0709 12:17:18.373777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.718610 (+344833us) service_pool.cc:224] Handling call
0709 12:17:18.727466 (+  8856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.082827 (+1355361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.991455 (+908628us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:20.991635 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2949652864}
W20260709 12:17:21.012478 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 130333us
W20260709 12:17:21.020805 18298 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.502s	user 0.015s	sys 0.004s
I20260709 12:17:21.021293 18298 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:21.023034 18443 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.026419 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:20.934635 20601 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:21.030867 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98998.9us
I20260709 12:17:21.034111 18371 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:21.045138 20546 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:21.053506 20576 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:20.969887 20606 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:17:21.056032 18448 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:21.056946 19039 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:21.071543 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.077548 20557 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.843s	user 0.014s	sys 0.019s
I20260709 12:17:21.078243 20284 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.033123 20452 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.047269 20359 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:21.079057 20452 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.078423 20239 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.083504 20549 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.084610 19990 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.084486 20386 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:21.085345 20580 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.013s
I20260709 12:17:21.085742 20580 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:21.086750 20576 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.088634 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:21.088971 20338 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.624s	user 0.014s	sys 0.008s
I20260709 12:17:21.089359 19043 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:21.092372 20585 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.005s
I20260709 12:17:21.092882 20585 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.993063 20602 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.094266 20239 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.094656 20239 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:21.094734 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1579) took 1269 ms. Trace:
I20260709 12:17:21.095180 18369 rpcz_store.cc:276] 0709 12:17:19.825825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.829127 (+  3302us) service_pool.cc:224] Handling call
0709 12:17:21.094518 (+1265391us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54000c8ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.094666 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2241328384}
I20260709 12:17:21.097494 20239 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.103295 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.103940 18371 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:21.109001 20579 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.109400 20579 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:21.110177 20579 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.097983 20601 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.115936 18369 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:21.117600 20511 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.117964 20511 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.120147 19038 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:17:21.118956 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.118808 18291 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
W20260709 12:17:21.119436 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 663413us
I20260709 12:17:21.124001 18371 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:21.125734 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.126441 18412 leader_election.cc:400] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:21.126838 18412 leader_election.cc:403] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:21.127205 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.127609 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 607) took 2269 ms. Trace:
I20260709 12:17:21.127934 19041 rpcz_store.cc:276] 0709 12:17:18.851397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.465011 (+1613614us) service_pool.cc:224] Handling call
0709 12:17:20.748083 (+283072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.754001 (+  5918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.119157 (+365156us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400454a50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.120301 (+  1144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650322688}
W20260709 12:17:21.128973 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.130560 18371 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.130972 18292 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:21.120059 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1501) took 1245 ms. Trace:
I20260709 12:17:21.086848 20386 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:21.159832 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.131753 18295 rpcz_store.cc:276] 0709 12:17:19.874520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.133980 (+259460us) service_pool.cc:224] Handling call
0709 12:17:20.134273 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:20.149777 (+ 15504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:20.149823 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:20.183803 (+ 33980us) log.cc:844] Serialized 26 byte log entry
0709 12:17:20.488153 (+304350us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:20.507800 (+ 19647us) log.cc:844] Serialized 14 byte log entry
0709 12:17:21.119758 (+611958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.119831 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:21.119874 (+    43us) raft_consensus.cc:1685] finished
0709 12:17:21.119886 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:21.120007 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":626745,"prepare.queue_time_us":19661,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":20066,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":187318,"threads_started":1,"wal-append.queue_time_us":250407}
I20260709 12:17:21.130170 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:21.165634 19043 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:21.165721 18444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1526) took 1379 ms (client timeout 1613 ms). Trace:
I20260709 12:17:21.159849 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:21.137033 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 247929us
I20260709 12:17:21.110707 20553 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.159371 18447 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:21.166234 18444 rpcz_store.cc:269] 0709 12:17:19.786601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.904050 (+117449us) service_pool.cc:224] Handling call
0709 12:17:21.156850 (+1252800us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003b15d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.165662 (+  8812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447661824}
I20260709 12:17:21.171060 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:21.168830 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.156315 20479 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.118712 20511 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.156065 20601 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:17:21.174347 20511 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.174216 20560 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:21.175261 20560 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:21.175647 20560 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.176046 20509 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:21.176465 20509 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.176779 20509 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:21.177156 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 521365us
I20260709 12:17:21.177870 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.178859 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.179414 20225 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:21.179597 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.180207 20225 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:21.180868 20225 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.166005 20570 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:17:21.181746 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:21.169891 20520 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:17:21.182725 20359 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:21.183075 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.183203 20359 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:21.159147 20555 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:21.170336 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:21.184190 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.166407 20609 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:21.184826 18298 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:21.168298 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:21.185447 20610 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.155s	user 0.000s	sys 0.004s
I20260709 12:17:21.185542 18300 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:21.184199 20359 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.192557 20519 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:21.192695 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.193120 20519 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:21.193207 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.194154 18296 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:21.193437 18295 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.172314 18293 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 1.
I20260709 12:17:21.196270 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:21.196856 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.197069 18300 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:21.197975 18298 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:21.194022 20519 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.199159 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.196874 18296 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:21.206485 20553 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.207486 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.208216 18293 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.209817 20515 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:21.211026 20528 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.662s	user 0.019s	sys 0.008s
W20260709 12:17:21.211623 19038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 609) took 2305 ms (client timeout 2187 ms). Trace:
I20260709 12:17:21.211452 20528 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:21.211980 19038 rpcz_store.cc:269] 0709 12:17:18.906103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.871218 (+965115us) service_pool.cc:224] Handling call
0709 12:17:21.211453 (+1340235us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400834fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.211558 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738230528}
I20260709 12:17:21.212646 18294 raft_consensus.cc:2368] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9.
I20260709 12:17:21.213532 20528 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.214818 20518 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.215500 20477 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.215931 18441 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:21.216428 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 469) took 2467 ms. Trace:
I20260709 12:17:21.216827 19035 rpcz_store.cc:276] 0709 12:17:18.749210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.761633 (+ 12423us) service_pool.cc:224] Handling call
0709 12:17:19.634823 (+873190us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57eee 000000000036eb39 00007f3c6ddcb1e0 00007f3c6de940c4 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:19.635049 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.974925 (+339876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.216190 (+1241265us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540071e8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:21.216348 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587748864}
I20260709 12:17:21.217164 18293 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:21.219476 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.220388 18295 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:21.220767 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.221455 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.223259 18293 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:21.215875 20477 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.163070 20579 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.221879 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.226303 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.227448 18300 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:21.228514 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 202149us
W20260709 12:17:21.229240 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50452.9us
I20260709 12:17:21.229758 18296 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:21.233549 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.239809 18296 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.240021 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.240690 18296 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:21.275492 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.275849 18296 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:21.276034 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.122862 20551 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:17:21.282594 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.276746 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:21.281635 18295 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.155689 20284 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:17:21.293111 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.302740 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:21.304582 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.306485 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.295456 18295 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:21.226730 20386 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.310146 18296 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:21.310729 18296 pending_rounds.cc:85] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 1
I20260709 12:17:21.311110 18296 pending_rounds.cc:107] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:21.311733 18296 raft_consensus.cc:2889] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:21.124900 20557 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.314150 18296 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:21.227779 20347 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:21.282464 20338 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.277855 20454 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:21.323166 18300 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:17:21.324064 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 814740us
I20260709 12:17:21.283030 20564 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.326418 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.294305 20511 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.301796 20585 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.317488 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.302806 20594 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.224195 20472 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:17:21.337558 19040 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:21.335769 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:21.313177 20528 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:21.340060 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 83652a40670f4224a56a5a1347b9d1a1: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:21.313448 20465 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.341281 18299 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:21.338941 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.344043 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.323727 20527 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:21.335407 20193 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.858s	user 0.016s	sys 0.006s
I20260709 12:17:21.346015 18299 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.350253 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.350984 18300 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:21.349660 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.353022 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.354555 18442 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:21.358834 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.359081 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.359783 18367 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.359886 20581 raft_consensus.cc:2764] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:17:21.360853 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.362954 18373 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:21.363859 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.367437 20555 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.367820 20555 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:21.355993 20471 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:17:21.368041 20518 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:21.368233 20555 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.368957 20537 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.018s
I20260709 12:17:21.348148 20225 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:17:21.369406 20537 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.370570 19041 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:21.371173 20528 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.371425 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:21.342662 20580 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.327307 20446 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:21.375139 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.375367 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.325513 20386 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.380354 18369 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:21.383016 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.346101 20477 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.334797 20338 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.384382 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.384441 20467 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.343637 20563 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:17:21.398821 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.384949 19039 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:17:21.385591 20578 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.016s
I20260709 12:17:21.380741 20193 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.384007 20527 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.336728 20454 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:21.394764 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 50797.8us
I20260709 12:17:21.343416 20594 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.426159 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.426896 18375 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:21.397166 20518 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:21.372481 20465 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.404255 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.406304 18367 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:21.388127 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:21.432644 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44655.7us
I20260709 12:17:21.379326 20564 raft_consensus.cc:2764] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:21.345908 20584 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:17:21.411590 18296 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:17:21.440372 18296 pending_rounds.cc:85] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 5
I20260709 12:17:21.440611 18296 pending_rounds.cc:107] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:17:21.440951 18296 raft_consensus.cc:2889] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:21.386775 20469 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:21.396056 20322 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.011s
I20260709 12:17:21.341717 20585 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.336637 20549 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:17:21.429209 20518 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.462718 20518 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:21.464192 20619 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:21.466420 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.466811 20594 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.467393 20469 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:21.408169 20577 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.582s	user 0.017s	sys 0.000s
I20260709 12:17:21.400017 20581 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.437901 20477 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.468264 20581 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:21.413527 20193 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.470311 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:21.479754 20193 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.480643 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.433354 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.436574 20454 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:21.482100 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48826.3us
I20260709 12:17:21.482425 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.482941 18291 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:21.438637 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.438138 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.413264 20537 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.452469 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:21.498023 19039 raft_consensus.cc:2376] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3. Re-sending same reply.
I20260709 12:17:21.458719 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.499352 18295 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:21.391566 20452 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:21.429062 20521 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:21.429399 20622 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:21.461362 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.461647 20607 raft_consensus.cc:2764] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:21.462005 20549 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:21.462129 20584 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.437800 20472 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.396323 18366 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:21.483983 18374 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:21.482951 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.501343 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.524384 18367 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:21.468734 20581 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.527245 18371 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:21.412319 20347 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.517092 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:21.533397 20446 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.534014 20446 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.535043 20446 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.555344 18366 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:21.500941 20468 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.452693 20509 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:21.558661 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 75743.6us
I20260709 12:17:21.543510 20322 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.562430 20576 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:21.468281 20469 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.563017 20557 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.502235 20622 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.556766 20454 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.564070 20338 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.508787 20549 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:21.516225 20584 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.567742 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.590456 18373 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:21.462221 20465 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:21.595299 20625 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:17:21.598754 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.637854 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.599294 18291 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:21.548655 20452 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.609937 18293 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:21.611711 20537 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:17:21.598227 20576 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.598721 20468 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.601030 20207 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:17:21.617401 20472 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:17:21.622705 20285 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:21.658550 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 305555us
I20260709 12:17:21.673483 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:21.669638 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44229.9us
I20260709 12:17:21.680963 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.601846 20322 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.648903 20576 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.714597 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.715634 20528 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.716121 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.717087 20528 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.717657 18264 leader_election.cc:400] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3.
I20260709 12:17:21.667141 20527 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:21.621460 20454 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:21.722121 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.726161 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.694012 20564 raft_consensus.cc:2764] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:21.723625 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.726253 18366 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:21.673096 20285 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:21.722012 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 205036us
I20260709 12:17:21.723618 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.713886 20563 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.738596 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.744828 20621 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:21.745272 20628 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:17:21.745572 20565 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:21.746217 20618 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:17:21.746703 20225 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:21.748383 20518 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.750808 20518 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:21.694516 20622 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.752503 20454 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.752588 20622 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.753250 20555 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.753376 20285 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.754513 20285 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.701812 20584 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.756814 20584 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.742484 20528 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:21.701606 20549 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.760886 20549 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:21.761166 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.762323 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.762706 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 481) took 1015 ms. Trace:
I20260709 12:17:21.728937 20527 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.765061 20452 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.768332 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 615) took 2860 ms. Trace:
I20260709 12:17:21.768698 19040 rpcz_store.cc:276] 0709 12:17:18.907860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.696288 (+1788428us) service_pool.cc:224] Handling call
0709 12:17:20.696568 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.337956 (+641388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.768064 (+430108us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54003a0554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:21.768266 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473709824}
I20260709 12:17:21.769078 20585 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.769989 20580 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.770385 20580 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.771168 20580 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.771860 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1537) took 1982 ms. Trace:
I20260709 12:17:21.772490 20580 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.775770 20509 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.776465 20322 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.776696 20509 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.777513 20509 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:21.777968 20509 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:21.778465 20509 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:21.778201 20193 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.765147 20468 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.779868 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.780999 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.781965 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.782644 18371 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:21.754912 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.784610 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.785493 18366 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:21.772184 18447 rpcz_store.cc:276] 0709 12:17:19.789367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.499970 (+710603us) service_pool.cc:224] Handling call
0709 12:17:21.159163 (+659193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.169710 (+ 10547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.548518 (+378808us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b540023dfa0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7fa02 00007f3c68e7bdd6 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:21.771682 (+223164us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54003b15d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.771805 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549134464}
I20260709 12:17:21.792743 20527 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:21.759979 20528 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:17:21.794792 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.795706 18294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1929) took 1309 ms (client timeout 1594 ms). Trace:
W20260709 12:17:21.796139 18294 rpcz_store.cc:269] 0709 12:17:20.486469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.510304 (+ 23835us) service_pool.cc:224] Handling call
0709 12:17:21.795533 (+1285229us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.795653 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580573952}
I20260709 12:17:21.798282 18297 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:21.799386 20193 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:21.799912 18194 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:21.721339 20581 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.801384 18194 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:21.802567 20581 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.803552 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.811141 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:21.815368 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 211108us
I20260709 12:17:21.763098 19044 rpcz_store.cc:276] 0709 12:17:20.747005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.795615 (+ 48610us) service_pool.cc:224] Handling call
0709 12:17:20.795907 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:20.815681 (+ 19774us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:20.871712 (+ 56031us) log.cc:844] Serialized 14 byte log entry
0709 12:17:21.406984 (+535272us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.410810 (+  3826us) log.cc:844] Serialized 26 byte log entry
0709 12:17:21.413118 (+  2308us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:21.428011 (+ 14893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.428206 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:21.762484 (+334278us) raft_consensus.cc:1685] finished
0709 12:17:21.762506 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:21.762645 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22573,"prepare.queue_time_us":69483,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":23556,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":13905,"thread_start_us":533034,"threads_started":2,"wal-append.queue_time_us":698228}
I20260709 12:17:21.803202 18194 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 5, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 5 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:21.817792 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.811143 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.820545 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.820454 20467 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:21.821866 20467 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.822328 20467 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.822620 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.822806 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.823659 20454 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.823858 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.823977 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.824477 18372 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:21.825229 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.825882 20609 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.826546 20609 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:21.826774 20609 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.827018 20609 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:21.827306 20207 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:21.827857 20207 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.826891 20472 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:21.824072 20454 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:21.830643 18292 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:17:21.831185 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 342) took 1408 ms. Trace:
I20260709 12:17:21.834022 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:21.839562 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:21.834399 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:21.834684 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:21.839370 18297 rpcz_store.cc:276] 0709 12:17:20.422556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.746684 (+324128us) service_pool.cc:224] Handling call
0709 12:17:21.798085 (+1051401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.798708 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.831002 (+ 32294us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000398d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:21.831129 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317204352}
W20260709 12:17:21.836452 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.836905 18372 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:21.837888 18447 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:21.838395 18371 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:21.839684 18296 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 2: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:21.842803 18372 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:21.842952 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.841612 18446 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:21.843060 18371 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:21.840631 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.844686 18372 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:21.841655 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:21.831370 20579 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.002s
I20260709 12:17:21.844805 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:21.859684 18293 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:21.850006 18371 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:21.863417 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:21.859414 20477 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:21.866031 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.866571 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.867023 19040 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:21.868080 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.854996 20560 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.869747 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.860092 18194 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:21.870463 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:21.885223 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:21.886484 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.888622 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:21.888962 19044 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:21.890395 18366 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:21.817094 20555 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:21.887917 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.891343 20225 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:21.859175 20193 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:17:21.894441 20207 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:21.894963 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:21.895635 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:21.851307 20611 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:21.899562 19042 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:21.898268 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:21.901415 20225 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.857518 20518 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:17:21.909003 20601 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:21.909169 20225 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.909776 20601 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:21.850947 20557 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:21.911453 20557 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:21.846274 20616 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:21.909523 20518 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:21.902372 20193 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:21.845502 20565 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:21.906409 18374 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:21.870033 20560 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:21.889990 20446 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:17:21.915133 20555 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.899959 20359 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.918313 20454 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.891638 20511 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.823212 20467 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:21.956445 20621 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.906121 20611 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:22.003978 20359 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.010664 20359 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:22.006283 20621 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:22.013116 20207 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:22.013643 20207 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:22.015967 20611 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.017390 20611 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:22.017648 20454 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.012866 20621 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:21.932068 20479 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:21.934034 20239 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.996008 18366 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:22.063692 20509 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.014686 20207 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.073071 18366 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:22.060158 20479 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:22.078905 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 316628us
I20260709 12:17:22.074381 20193 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:22.076081 20621 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.006860 20557 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:22.183485 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 313538us
I20260709 12:17:22.078317 18368 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:22.121418 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.192914 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.194901 18299 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:22.199294 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.199738 18291 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:22.086040 20477 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:22.196254 18366 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.210513 18366 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.208400 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.208652 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.211719 18299 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:22.204505 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.212992 18297 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:22.213515 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.213402 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.214184 18294 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:22.214257 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 356692us
W20260709 12:17:22.185822 20632 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.441s	user 0.004s	sys 0.002s
I20260709 12:17:22.209645 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:22.212404 18292 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:22.221913 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 402101us
I20260709 12:17:22.223877 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:22.224668 18299 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.227339 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:22.222501 18338 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 2.242s (SENT)
W20260709 12:17:22.209103 20469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.023s	sys 0.051s
I20260709 12:17:22.206967 20563 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:22.235446 19042 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:22.235678 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.200794 20584 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:22.080914 20565 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.242130 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:22.259851 19043 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:22.206676 20578 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:22.245021 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:22.264829 18297 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:22.209522 20570 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.248662 18375 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:22.234869 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:22.267284 18294 raft_consensus.cc:2376] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3. Re-sending same reply.
W20260709 12:17:22.243667 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.261561 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.262518 18299 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:22.268702 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:22.231889 20607 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.247476 20469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.023s	sys 0.051s
I20260709 12:17:22.270439 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.232792 20468 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:22.278064 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 379671us
I20260709 12:17:22.257083 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:22.285228 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.286821 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:22.278596 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:22.221091 20479 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.287494 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:22.374626 18297 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:22.288491 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:22.298418 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.366009 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:22.422217 18298 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:22.422444 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.423511 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:22.414443 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.379211 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 351) took 1036 ms. Trace:
I20260709 12:17:22.288780 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:22.259685 20563 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.447273 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.503589 18298 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:22.423687 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:22.429703 18300 rpcz_store.cc:276] 0709 12:17:21.342642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.359636 (+ 16994us) service_pool.cc:224] Handling call
0709 12:17:21.360029 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.400839 (+ 40810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.400877 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.401355 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:17:21.613332 (+211977us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:21.935083 (+321751us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:22.006665 (+ 71582us) log.cc:844] Serialized 14 byte log entry
0709 12:17:22.006957 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.007065 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.286875 (+279810us) raft_consensus.cc:1685] finished
0709 12:17:22.302904 (+ 16029us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.379099 (+ 76195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56115,"prepare.queue_time_us":463966,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":491,"raft.queue_time_us":54844,"raft.run_cpu_time_us":4239,"raft.run_wall_time_us":44986,"spinlock_wait_cycles":40833408,"thread_start_us":541200,"threads_started":2,"wal-append.queue_time_us":327861}
I20260709 12:17:22.288215 20322 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:22.508008 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 220567us
I20260709 12:17:22.287539 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.392992 20469 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.506022 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 140134us
I20260709 12:17:22.269552 20465 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.489629 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.516328 18443 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:22.517387 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.366611 20640 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:22.487886 20639 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.415s	user 0.001s	sys 0.001s
W20260709 12:17:22.509838 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 222381us
I20260709 12:17:22.510448 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:22.283468 20193 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.518111 18443 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:22.508180 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.358062 20555 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.217s
I20260709 12:17:22.355388 19035 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:22.548115 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.501187 20636 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.440s	user 0.001s	sys 0.000s
I20260709 12:17:22.519062 20285 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:22.525391 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:22.520275 20564 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.277519 20578 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.378573 20549 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.536926 18291 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.536790 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:22.538233 18338 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.594s (SENT)
I20260709 12:17:22.282099 20607 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.281442 20570 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.542309 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.576238 18445 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.552651 20647 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.561239 18375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80bc05a330c64471aaa556af75bea031"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:22.579823 18443 pending_rounds.cc:85] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 3
I20260709 12:17:22.358619 20477 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.525079 20579 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:22.565434 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.579931 18375 raft_consensus.cc:606] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:22.586923 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.519471 20521 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:22.527387 20509 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:22.571908 18449 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:22.573385 18291 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:22.564646 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.530650 19036 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:22.265530 20584 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.580140 18443 pending_rounds.cc:107] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:22.596197 18443 raft_consensus.cc:2889] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:22.603277 20579 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:22.561785 20642 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 12:17:22.583628 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1591) took 1158 ms. Trace:
I20260709 12:17:22.589097 18293 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:22.591257 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 977c0af9c8e94119b3d18a0ee7401458: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:22.593415 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:22.672356 18445 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.531051 20465 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.594815 20521 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.682090 20207 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:22.525275 20469 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:22.611773 20653 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:17:22.684353 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 353) took 1307 ms. Trace:
I20260709 12:17:22.684662 20469 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:22.283308 20468 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.684736 18295 rpcz_store.cc:276] 0709 12:17:21.377334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.503902 (+126568us) service_pool.cc:224] Handling call
0709 12:17:21.545043 (+ 41141us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.559345 (+ 14302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.559391 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.583217 (+ 23826us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.083627 (+500410us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:22.083688 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.083773 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.684135 (+600362us) raft_consensus.cc:1685] finished
0709 12:17:22.684155 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.684297 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14499,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":439630,"threads_started":1,"wal-append.queue_time_us":663346}
I20260709 12:17:22.686604 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:22.686764 20468 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:22.688153 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:22.688453 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:22.688756 18449 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:22.689114 18374 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:22.534742 20640 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.691361 20640 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:22.667946 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.668195 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 129976us
I20260709 12:17:22.568710 20519 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:22.669401 18367 rpcz_store.cc:276] 0709 12:17:21.424929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.525005 (+100076us) service_pool.cc:224] Handling call
0709 12:17:21.525389 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.525658 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.525695 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.526243 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:17:21.738533 (+212290us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:21.738566 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.738641 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.583429 (+844788us) raft_consensus.cc:1685] finished
0709 12:17:22.583450 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.583571 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":211912,"threads_started":1,"wal-append.queue_time_us":396339}
W20260709 12:17:22.591408 20652 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:17:22.679646 20386 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:22.680419 20465 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:22.682233 20578 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:22.599963 20648 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 12:17:22.700486 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.711901 18445 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:22.713438 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:22.669216 20347 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.667670 20643 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.675473 20576 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.951s	user 0.012s	sys 0.002s
W20260709 12:17:22.675896 20654 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.283s	user 0.000s	sys 0.006s
I20260709 12:17:22.669891 20472 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.667520 20537 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:22.716249 18366 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.002s
I20260709 12:17:22.727818 18366 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:22.728163 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:22.728706 20645 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.181s	user 0.002s	sys 0.003s
I20260709 12:17:22.745307 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.727599 20578 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:22.816246 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 71265.1us
I20260709 12:17:22.821740 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.590639 20647 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.823233 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.731371 20347 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:22.849464 20347 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:22.852452 20621 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.001s
I20260709 12:17:22.729214 20576 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.827476 20551 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:22.853627 20551 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.854218 20551 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:22.854614 20576 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:22.856441 20647 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:22.856976 20578 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:22.857899 20578 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.827675 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:22.820187 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74958.1us
I20260709 12:17:22.859776 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:22.738271 20465 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.832521 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.836005 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 459) took 1003 ms. Trace:
W20260709 12:17:22.836309 20655 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.330s	user 0.001s	sys 0.001s
I20260709 12:17:22.836647 20584 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:22.731575 20643 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.837343 20579 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:22.838510 20622 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.085s	user 0.006s	sys 0.008s
W20260709 12:17:22.843566 20609 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.016s	user 0.008s	sys 0.005s
I20260709 12:17:22.843778 20607 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:22.731040 20472 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.844578 20563 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:22.845011 20239 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.771s	user 0.007s	sys 0.004s
W20260709 12:17:22.846469 20454 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.001s
W20260709 12:17:22.848435 20560 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.001s
W20260709 12:17:22.848511 20611 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.004s
I20260709 12:17:22.815927 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:22.827867 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.861229 18292 raft_consensus.cc:2410] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:22.860690 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:22.910521 18369 raft_consensus.cc:2410] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:22.863511 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1592) took 1272 ms. Trace:
I20260709 12:17:22.865037 18445 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:22.819947 20519 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.834077 20549 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.864306 18440 rpcz_store.cc:276] 0709 12:17:21.832313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.834214 (+  1901us) service_pool.cc:224] Handling call
0709 12:17:21.858835 (+ 24621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.956145 (+ 97310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.956189 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.070363 (+114174us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.279773 (+209410us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:22.279848 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.279944 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.835733 (+555789us) raft_consensus.cc:1685] finished
0709 12:17:22.835764 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.835929 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72322,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":73381,"thread_start_us":296764,"threads_started":2,"wal-append.queue_time_us":672230}
I20260709 12:17:22.855149 20551 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:22.826195 20650 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.852487 20555 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:22.842935 20521 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:22.883963 18447 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:22.869184 20601 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:22.963344 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.026s	sys 0.037s
W20260709 12:17:22.984076 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.027s	sys 0.037s
W20260709 12:17:22.984643 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 1.08668e+06us
I20260709 12:17:22.826282 20666 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:22.961560 18373 rpcz_store.cc:276] 0709 12:17:21.591253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.849895 (+258642us) service_pool.cc:224] Handling call
0709 12:17:21.850338 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:21.850798 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:21.850841 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:21.851581 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.205448 (+353867us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:22.205497 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.205581 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:22.757669 (+552088us) raft_consensus.cc:1685] finished
0709 12:17:22.757698 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:22.863435 (+105737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1415,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":353424,"threads_started":1,"wal-append.queue_time_us":690926}
W20260709 12:17:22.965698 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 105087us
I20260709 12:17:22.876430 20509 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:22.969455 18449 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:22.878715 20644 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:22.962170 20322 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.882365 20570 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:22.990267 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:22.959913 20563 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:22.883050 20609 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.968981 20239 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:22.996111 18441 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
W20260709 12:17:22.997257 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 137466us
W20260709 12:17:22.996747 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 495014us
I20260709 12:17:22.998551 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:22.884606 20673 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:23.011639 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:23.013223 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:22.970270 20551 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:22.983397 20564 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:22.871023 20465 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:22.985983 20622 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.970835 20519 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.019965 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.004694 20549 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:23.007367 20570 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.067701 18446 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260709 12:17:23.060724 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51874.2us
W20260709 12:17:23.063529 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 43677.2us
I20260709 12:17:22.989050 20601 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.023680 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:22.971957 19036 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:22.982849 20650 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:23.010118 20668 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:17:23.087486 18373 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:23.006075 20563 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.094362 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.066627 20549 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.058393 20584 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.095727 20659 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:17:22.999576 20509 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.149273 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.194068 20643 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.093086 20666 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:23.067826 20454 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:23.164083 20671 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:17:23.214967 20549 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:23.097812 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.230410 19038 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.097108 19044 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:23.154052 20681 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.099908 20519 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.068809 20672 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:17:23.223986 20643 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.998127 20322 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:23.247319 20678 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:17:23.147923 20551 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:23.189688 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 203975us
I20260709 12:17:23.249060 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.021800 20607 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.251058 20607 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.251459 18448 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:23.251670 20607 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:23.252691 20675 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:17:23.253088 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1931) took 1063 ms. Trace:
I20260709 12:17:23.253679 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1540) took 1023 ms. Trace:
I20260709 12:17:23.066699 20621 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.254089 18300 rpcz_store.cc:276] 0709 12:17:22.230009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.504770 (+274761us) service_pool.cc:224] Handling call
0709 12:17:22.505105 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.513861 (+  8756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.513892 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.525104 (+ 11212us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.823825 (+298721us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:22.823874 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.823957 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.253461 (+429504us) raft_consensus.cc:1685] finished
0709 12:17:23.253481 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.253618 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":577,"raft.queue_time_us":13953,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":21327,"thread_start_us":287639,"threads_started":1,"wal-append.queue_time_us":482968}
I20260709 12:17:23.255511 20621 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.255864 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:23.256565 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1534) took 1040 ms. Trace:
I20260709 12:17:23.256966 18296 rpcz_store.cc:276] 0709 12:17:22.216171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.217307 (+  1136us) service_pool.cc:224] Handling call
0709 12:17:23.255277 (+1037970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.255456 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.255485 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:23.255557 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:23.255598 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.255652 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.256369 (+   717us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540011f650. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.256474 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385909632}
I20260709 12:17:23.258225 20621 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:23.196275 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.253433 18293 rpcz_store.cc:276] 0709 12:17:22.189797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.189937 (+   140us) service_pool.cc:224] Handling call
0709 12:17:23.252890 (+1062953us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540011f654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.253010 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866679680}
I20260709 12:17:23.066249 20622 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:23.093770 18368 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1455) took 1642 ms (client timeout 1534 ms). Trace:
W20260709 12:17:23.263185 18368 rpcz_store.cc:269] 0709 12:17:21.450889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.466102 (+ 15213us) service_pool.cc:224] Handling call
0709 12:17:23.093458 (+1627356us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400395b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.093689 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904825216}
I20260709 12:17:23.102468 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.264523 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 461) took 1431 ms. Trace:
I20260709 12:17:23.264870 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.210314 18298 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:23.184449 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.266726 19035 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:23.195812 20518 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.258963 20621 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.268548 20518 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.268707 20621 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.270592 20518 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.185806 20555 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:23.271826 20650 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.068005 20570 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.219916 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:23.200896 20576 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.065753 20673 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.229303 18373 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:23.214637 20564 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.058125 20477 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.219674 20674 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.233851 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.223527 20454 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.295306 18375 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:23.095883 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.243135 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1939) took 1042 ms. Trace:
I20260709 12:17:23.298341 18291 rpcz_store.cc:276] 0709 12:17:22.200670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.209426 (+  8756us) service_pool.cc:224] Handling call
0709 12:17:23.242875 (+1033449us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b540004cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.243059 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725732608}
W20260709 12:17:23.241593 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.219133 20584 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.244136 20472 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.261700 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.318166 18448 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:23.262228 20521 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:23.262969 18415 connection.cc:256] RPC call timeout handler was delayed by -1.1459236350001447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2277s for a response
I20260709 12:17:23.235510 20670 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:23.272830 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1538) took 1050 ms. Trace:
I20260709 12:17:23.272543 20616 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.264896 18441 rpcz_store.cc:276] 0709 12:17:21.832871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.841435 (+  8564us) service_pool.cc:224] Handling call
0709 12:17:23.264359 (+1422924us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400422f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.264457 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546538368}
W20260709 12:17:23.290253 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56517.9us
I20260709 12:17:23.331827 18297 rpcz_store.cc:276] 0709 12:17:22.222232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.504669 (+282437us) service_pool.cc:224] Handling call
0709 12:17:22.531063 (+ 26394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.542128 (+ 11065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.542176 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.542736 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.717669 (+174933us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:22.718117 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.718477 (+   360us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.272546 (+554069us) raft_consensus.cc:1685] finished
0709 12:17:23.272577 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.272749 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17215,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":16353,"thread_start_us":174306,"threads_started":1,"wal-append.queue_time_us":461991}
I20260709 12:17:23.272014 20555 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.304708 20560 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.241622 20681 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.293632 20673 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:23.272986 18440 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:23.162645 20601 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.307178 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.000068 20644 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.308519 20553 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.305889 20477 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.309909 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.311287 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:23.312773 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.244606 20565 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.318972 18293 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.319139 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:23.266093 20677 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:17:23.287930 20570 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.328110 19041 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:17:23.330509 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 65085.8us
I20260709 12:17:23.384429 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.288652 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.294605 20509 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.339146 18300 raft_consensus.cc:2376] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2. Re-sending same reply.
I20260709 12:17:23.302790 20454 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.260735 20643 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.333159 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.309223 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.308709 20285 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.365017 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.334661 20621 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.339794 20616 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:23.323671 20666 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.383383 19035 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:23.331682 20670 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:23.363152 20347 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:17:23.370533 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.373446 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1945) took 1087 ms. Trace:
I20260709 12:17:23.373381 18448 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:23.383248 20684 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.380759 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.359606 20560 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.303109 20564 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.338266 20519 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:17:23.437592 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.367337 20667 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:23.418344 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85284.6us
I20260709 12:17:23.368650 20518 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.294301 20576 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.303031 20674 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.366163 20553 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.448482 20685 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:17:23.426472 18293 rpcz_store.cc:276] 0709 12:17:22.286339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.289848 (+1003509us) service_pool.cc:224] Handling call
0709 12:17:23.318824 (+ 28976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.373225 (+ 54401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.373393 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:23.449926 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.450419 20553 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.450518 18294 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.383286 20688 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.451583 20688 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.446580 20518 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.453239 20239 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:23.453619 20239 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:23.453853 20518 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.454398 20518 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:23.430904 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:23.433550 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1542) took 1199 ms. Trace:
I20260709 12:17:23.434273 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.423398 20285 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.433890 20560 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.435220 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.435832 20697 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:23.454416 20239 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.466256 20697 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.466711 20697 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:23.466869 20239 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.436823 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1943) took 1178 ms. Trace:
I20260709 12:17:23.438419 18449 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:23.463019 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.473285 19044 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:23.435313 20644 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.422914 20356 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.467274 20285 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.389216 20578 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:17:23.441256 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.422875 20509 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.425043 20193 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.387408 20565 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.535171 18291 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:23.536275 20565 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.424283 20621 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.536950 20621 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.538507 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.427803 20579 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.448554 20563 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.452068 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.450431 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:23.428321 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 75467.7us
I20260709 12:17:23.453811 20688 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.455778 18295 rpcz_store.cc:276] 0709 12:17:22.233975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.693745 (+459770us) service_pool.cc:224] Handling call
0709 12:17:22.694171 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.695009 (+   838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.695041 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.695474 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.873114 (+177640us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:22.873166 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.873250 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.433310 (+560060us) raft_consensus.cc:1685] finished
0709 12:17:23.433328 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.433480 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":177282,"threads_started":1,"wal-append.queue_time_us":288204}
I20260709 12:17:23.473730 20519 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.469753 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.473208 18296 rpcz_store.cc:276] 0709 12:17:22.258197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.258041 (+999844us) service_pool.cc:224] Handling call
0709 12:17:23.258320 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.258526 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:23.330776 (+ 72250us) log.cc:844] Serialized 14 byte log entry
0709 12:17:23.436290 (+105514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:23.436367 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:23.436440 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.436558 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.436747 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15517,"prepare.queue_time_us":157933,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":20709,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":156643,"threads_started":2}
I20260709 12:17:23.468758 18445 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:23.467345 20695 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.518808 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1537) took 1298 ms. Trace:
I20260709 12:17:23.442452 20650 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:17:23.522480 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 81292us
I20260709 12:17:23.525044 20193 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:23.525995 20690 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:17:23.526975 20648 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:23.527063 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 6c5d2fad2696483cae18eab3ba337ec9: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:23.507762 20518 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:23.507375 19035 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.506803 20356 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.529053 20477 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.529624 20584 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:23.530581 20689 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:17:23.534219 20622 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.431493 20666 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.577731 20356 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.584012 20706 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.584676 20666 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.584599 20706 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.586656 20706 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.541175 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:23.587397 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 46344.5us
I20260709 12:17:23.563982 20695 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.588546 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.588966 20193 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.547729 19043 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
W20260709 12:17:23.589704 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.590361 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.590951 19043 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:23.589829 20695 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.589677 20706 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.591776 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.592252 20454 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.592855 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.592937 20616 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.551796 18292 rpcz_store.cc:276] 0709 12:17:22.220153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.221671 (+  1518us) service_pool.cc:224] Handling call
0709 12:17:23.518624 (+1296953us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54000393d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:23.518751 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451086720}
I20260709 12:17:23.594169 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.594714 18292 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:23.595518 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.596484 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.597384 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.550952 20579 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.598073 18292 raft_consensus.cc:2410] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:17:23.552209 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 110400us
I20260709 12:17:23.598820 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.549151 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.598986 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:23.548341 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.564049 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.444932 20667 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.515455 20644 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.433122 20684 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.534973 20607 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.547554 20519 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:23.479123 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.587492 20477 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:23.552309 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:23.598696 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.561079 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:23.539114 20621 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.600499 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.564019 20688 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.617336 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.671015 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.539758 20509 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.675415 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:23.705555 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.677587 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.718111 20516 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.719053 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.719767 18368 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:23.720453 20607 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.713866 18296 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:23.721748 18293 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:23.670387 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118155us
I20260709 12:17:23.722330 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.669847 18292 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:23.679889 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.724061 18371 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:23.671800 18442 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:23.724212 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.720734 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.590395 20576 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.730432 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.730551 20479 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:23.731578 20650 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.731822 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.732548 20601 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:23.733812 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.734222 20696 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.734472 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.735734 20555 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.721300 20648 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.736104 20555 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.736425 20648 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.736537 20555 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.736752 20509 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.723201 18295 raft_consensus.cc:2410] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:23.738204 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.739467 18295 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:23.740765 20667 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.741482 20551 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.741664 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.742357 18375 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:23.743854 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.595075 20454 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.674340 20519 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.745766 20609 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.746124 20356 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.747690 20616 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.725530 18370 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:23.727401 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.731959 20650 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.736841 20648 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.745558 20684 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.747984 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:23.783903 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.690122 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.784880 18374 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:23.679196 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:23.785380 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.785805 18367 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
W20260709 12:17:23.706197 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 107291us
W20260709 12:17:23.783077 20467 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.766s	user 0.017s	sys 0.033s
I20260709 12:17:23.747431 18295 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:23.697752 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.795130 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.680637 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.708290 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:23.751458 19043 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:23.796253 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.797034 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.722882 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:23.669494 20477 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.778545 20650 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.765616 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.522087 20670 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.780534 20709 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:23.780323 20284 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:23.606750 20673 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:17:23.781239 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 67a1783f49244bd5ba98721205d1ff6e: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:23.781543 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:23.748978 18293 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.685184 20644 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.829083 20673 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.830826 20644 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.726153 18292 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 2.
I20260709 12:17:23.832669 18292 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:23.734827 20696 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.787003 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.755684 20681 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.789234 20356 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.680265 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.603057 20579 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.672699 20193 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.793434 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.793815 18367 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:23.798363 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:23.798657 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 90462.5us
I20260709 12:17:23.796204 18366 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:23.759445 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:23.803092 18371 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:23.748660 20674 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.783866 20509 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.609146 20285 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.215s
W20260709 12:17:23.794317 20467 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.777s	user 0.018s	sys 0.033s
I20260709 12:17:23.790876 20616 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.725859 20549 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.782935 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:23.865195 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 82358.3us
I20260709 12:17:23.795630 20555 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.820616 20709 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.801724 20477 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:23.820870 20704 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.797150 20564 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.841058 18373 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:23.807688 20609 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:23.853055 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93728.6us
I20260709 12:17:23.870154 20563 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:23.887914 18296 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.889998 20563 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:23.890116 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:23.890589 20579 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.890003 20465 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:23.891579 20467 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:23.892158 20704 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.868032 20674 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:17:23.890580 20563 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:23.886094 20644 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.893921 20644 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.894496 20704 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.853035 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:23.895539 18366 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.896010 20644 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:23.851775 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:23.889428 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:23.855118 18375 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:17:23.897351 18367 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:23.891633 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.861433 18440 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:23.777432 20697 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.852545 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.828181 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.866235 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.869107 19043 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:23.835583 20284 consensus_queue.cc:1243] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:23.849057 18368 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:23.891172 20579 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.898437 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.902567 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.903522 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:23.898738 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46336.6us
I20260709 12:17:23.850776 20565 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.831140 20519 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.856287 20239 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.861011 20454 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.928061 19036 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:23.939370 20697 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.872090 18444 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.904665 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.942444 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:23.912458 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:23.909363 18299 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:23.920279 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:23.946552 18366 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
W20260709 12:17:23.947327 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 43864.1us
I20260709 12:17:23.948360 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:23.906502 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.901710 20704 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:23.933463 20667 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:23.933737 20467 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:23.909720 20609 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:23.936249 19035 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:23.937511 20519 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.939440 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.722878 20688 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.946784 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:23.950512 19041 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:17:23.950608 18415 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.534s (SENT)
I20260709 12:17:23.907044 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:23.951601 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:23.905484 20706 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.904702 20452 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.884285 20696 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.880344 20549 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.888984 20709 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.929881 20564 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:23.909557 20347 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:17:23.869966 20616 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.943325 20665 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:23.994699 18264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.014s	sys 0.009s
I20260709 12:17:23.995836 20609 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.996963 20705 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:23.997421 20705 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:23.998281 20705 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.989372 20564 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:24.000735 20564 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.000684 20665 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:24.003310 20665 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.005615 20477 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.968262 20471 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.012415 20471 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.965003 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:24.013863 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48927.2us
W20260709 12:17:24.014638 20711 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:17:24.015115 20284 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:24.015676 20284 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:24.016502 20284 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.914876 20644 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.937602 20579 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.865432 20509 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.011426 20616 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:23.968535 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:24.026940 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58507.6us
I20260709 12:17:24.027045 20709 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.027607 20709 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.027974 20709 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.028141 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.029241 20584 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.029589 20584 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.029902 20584 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.949810 20673 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.941704 20648 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:23.970934 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:24.047495 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 76633.3us
I20260709 12:17:24.048471 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:24.049796 20616 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.954792 20386 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:23.953583 20454 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.953429 20697 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.930944 20193 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:17:23.928115 20695 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.943754 20565 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.946691 20239 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:23.972124 20621 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:24.001658 20452 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:23.995070 18264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.014s	sys 0.009s
I20260709 12:17:23.909076 20465 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:23.959498 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:23.968261 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:24.065166 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 680766us
I20260709 12:17:24.023239 19043 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:24.065663 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 106270us
I20260709 12:17:24.025969 20688 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.071368 20452 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.031538 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.074831 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.075601 18366 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:24.076602 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.077212 18366 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.031462 18372 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:23.951325 20657 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.058511 20673 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.989079 20713 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:24.054877 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.056687 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.022851 20644 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.096552 18375 raft_consensus.cc:2410] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:24.047821 20622 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.063731 20621 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.068840 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.107930 18442 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:24.051337 20454 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:24.051896 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.056350 20697 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:24.091933 19043 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:24.090862 20673 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.074991 20193 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.062165 20616 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:17:24.090197 20477 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.067914 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.099387 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.057395 20705 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:24.119894 18447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1550) took 1852 ms (client timeout 2114 ms). Trace:
I20260709 12:17:24.140643 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.120177 18448 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:24.093920 20696 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:24.104457 20465 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.119208 18367 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:24.092801 20713 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.090971 20657 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:24.144191 20679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.010s	sys 0.003s
I20260709 12:17:24.097649 20644 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.145824 20673 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.159261 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:24.146515 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 195915us
I20260709 12:17:24.062446 20695 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.098321 20386 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.151678 18448 pending_rounds.cc:85] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 3
I20260709 12:17:24.119017 20621 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:24.151491 18447 rpcz_store.cc:269] 0709 12:17:22.268074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.268486 (+   412us) service_pool.cc:224] Handling call
0709 12:17:24.119704 (+1851218us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b5400258ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.119846 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412645632}
I20260709 12:17:24.152446 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:24.152918 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53665.3us
I20260709 12:17:24.140470 20454 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:24.064157 20560 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.001s
I20260709 12:17:24.131347 20706 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:24.143054 20697 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:24.147584 20570 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.003s
I20260709 12:17:24.052603 20674 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:17:24.169936 18442 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:24.169919 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:24.100992 20563 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:24.171985 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.169312 18366 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:24.172473 18448 pending_rounds.cc:107] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:24.154964 20521 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:24.157611 20679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.010s	sys 0.003s
I20260709 12:17:24.144095 20207 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:17:24.120344 20622 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.154948 20616 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.155264 20657 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.256450 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1548) took 1990 ms. Trace:
I20260709 12:17:24.256839 18444 rpcz_store.cc:276] 0709 12:17:22.266560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.266709 (+   149us) service_pool.cc:224] Handling call
0709 12:17:22.266981 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.940522 (+1673541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.256247 (+315725us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54002594d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:24.256396 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3687485312}
I20260709 12:17:24.257956 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.258689 18444 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:24.176204 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:24.190830 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.263540 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:24.198993 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.263592 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:24.264564 18445 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:24.265497 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.199771 18194 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:24.200222 18298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1942) took 1958 ms (client timeout 2093 ms). Trace:
W20260709 12:17:24.269745 18298 rpcz_store.cc:269] 0709 12:17:22.241648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.504294 (+262646us) service_pool.cc:224] Handling call
0709 12:17:24.200051 (+1695757us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.200164 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181557120}
I20260709 12:17:24.205045 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 1996) took 1129 ms. Trace:
I20260709 12:17:24.201817 18297 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:24.209901 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1949) took 1193 ms. Trace:
I20260709 12:17:24.277066 18300 rpcz_store.cc:276] 0709 12:17:23.016807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.255632 (+238825us) service_pool.cc:224] Handling call
0709 12:17:24.209698 (+954066us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b5400090454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.209828 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190005504}
I20260709 12:17:24.278398 19040 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:24.212785 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:24.280364 18440 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.282215 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:24.156908 20713 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:24.235579 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1549) took 1968 ms. Trace:
I20260709 12:17:24.284088 18446 rpcz_store.cc:276] 0709 12:17:22.266943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.267369 (+   426us) service_pool.cc:224] Handling call
0709 12:17:22.267735 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.086401 (+818666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.235319 (+1148918us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400423454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:24.235519 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1838875520}
I20260709 12:17:24.236896 18373 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:24.246757 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:24.199455 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:24.287142 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 87760.6us
I20260709 12:17:24.287362 19040 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:24.287906 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.288645 18194 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 2, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 2 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:24.220168 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59426us
I20260709 12:17:24.290233 18194 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:24.291934 18415 leader_election.cc:336] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.967s (SENT)
I20260709 12:17:24.292639 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:24.292805 18369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1449) took 2889 ms (client timeout 2728 ms). Trace:
I20260709 12:17:24.241283 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.271910 18443 rpcz_store.cc:276] 0709 12:17:23.075808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.250009 (+174201us) service_pool.cc:224] Handling call
0709 12:17:23.250322 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:23.250525 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:23.330085 (+ 79560us) log.cc:844] Serialized 14 byte log entry
0709 12:17:24.204511 (+874426us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18003b49a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287
0709 12:17:24.204629 (+   118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:24.204704 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:24.204771 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.204854 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.204983 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133478,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":43,"raft.queue_time_us":106933,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":19452,"spinlock_wait_cycles":3691904,"thread_start_us":168141,"threads_started":3,"wal-append.queue_time_us":35166}
W20260709 12:17:24.293159 18369 rpcz_store.cc:269] 0709 12:17:21.403261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.404032 (+   771us) service_pool.cc:224] Handling call
0709 12:17:24.292615 (+2888583us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002f8854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.292732 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3399633152}
I20260709 12:17:24.216980 18448 raft_consensus.cc:2889] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:24.298060 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:24.298765 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.298637 18369 raft_consensus.cc:2376] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4. Re-sending same reply.
I20260709 12:17:24.293673 18449 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:24.298594 18443 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:24.299510 19038 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:24.301991 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.302347 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.302748 18299 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:24.302939 18298 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:24.303135 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.306704 18372 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:24.303843 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.307456 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.308784 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:24.301008 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:24.307961 18372 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:24.312117 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.312458 18198 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 291) took 9144 ms. Trace:
I20260709 12:17:24.314447 18367 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:24.315653 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.316673 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.299407 18194 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:24.318224 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.318745 18291 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.320319 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.175645 20728 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.224292 20557 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.337288 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:24.337810 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 270553us
I20260709 12:17:24.338655 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.323987 18295 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:24.324709 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 154886us
I20260709 12:17:24.230686 20670 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.203617 20706 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:24.201570 20585 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:24.187172 20555 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.208043 20674 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.282598 20682 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:24.342232 20716 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:17:24.333724 20537 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:24.334138 18291 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:24.311295 20551 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:24.219880 20622 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:24.371337 20720 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:17:24.201081 20647 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:24.335316 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.336647 18375 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:24.156808 20696 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.324604 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.345479 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:24.345556 18440 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:24.217247 20521 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:24.349601 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.350426 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.243615 20643 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.305411 20579 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:24.307031 20679 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.355671 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.402486 20521 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.403319 20679 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.403767 20521 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:24.404193 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.404954 18373 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:24.405347 20679 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.405900 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.406950 18373 raft_consensus.cc:2410] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:24.363654 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 496) took 1026 ms. Trace:
I20260709 12:17:24.215922 20697 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.203470 20560 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.207578 20570 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.374205 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.217806 20563 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:24.398105 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 59612.1us
I20260709 12:17:24.432601 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.341252 20509 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.397532 20537 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.226864 20454 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.436286 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.437011 18375 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.412662 20576 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:24.438720 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93303.8us
W20260709 12:17:24.413566 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 89048us
I20260709 12:17:24.409251 20732 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.410193 20647 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:24.408059 20551 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:24.369815 20585 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:24.427476 18366 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:24.399546 20650 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.001s
W20260709 12:17:24.429345 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52831.4us
I20260709 12:17:24.429450 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1558) took 1168 ms. Trace:
I20260709 12:17:24.433530 18370 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:24.472815 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.472980 18448 rpcz_store.cc:276] 0709 12:17:23.260567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.460466 (+199899us) service_pool.cc:224] Handling call
0709 12:17:24.119826 (+659360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.294758 (+174932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.429207 (+134449us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400258ad0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.429363 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515574144}
I20260709 12:17:24.431366 20643 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.417207 18369 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:24.476313 18369 pending_rounds.cc:85] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 1
I20260709 12:17:24.476558 18369 pending_rounds.cc:107] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:24.476804 18369 raft_consensus.cc:2889] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:24.449452 18441 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:24.394193 20674 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.341293 20728 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.312827 18198 rpcz_store.cc:276] 0709 12:17:15.153813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.154286 (+   473us) service_pool.cc:224] Handling call
0709 12:17:24.297352 (+9143066us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:18.824058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:18.845715 (+ 21657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:18.845791 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:17:18.845834 (+    43us) tablet.cc:662] Decoding operations
0709 12:17:18.847814 (+  1980us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:18.847930 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:18.847969 (+    39us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:17:19.039884 (+191915us) tablet.cc:701] Row locks acquired
0709 12:17:19.039926 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:17:19.094143 (+ 54217us) write_op.cc:270] Start()
0709 12:17:19.167742 (+ 73599us) write_op.cc:276] Timestamp: P: 1783599439048607 usec, L: 0
0709 12:17:19.197943 (+ 30201us) op_driver.cc:348] REPLICATION: starting
0709 12:17:19.428256 (+230313us) log.cc:844] Serialized 4397 byte log entry
0709 12:17:20.679885 (+1251629us) op_driver.cc:464] REPLICATION: finished
0709 12:17:20.857443 (+177558us) write_op.cc:301] APPLY: starting
0709 12:17:20.857594 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:20.857871 (+   277us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:20.858001 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:20.989194 (+131193us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:20.989459 (+   265us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:17:20.989507 (+    48us) write_op.cc:312] APPLY: finished
0709 12:17:20.989888 (+   381us) log.cc:844] Serialized 83 byte log entry
0709 12:17:20.990084 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:20.990248 (+   164us) write_op.cc:454] Released schema lock
0709 12:17:20.990300 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:20.990655 (+   355us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:20.990686 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":503,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":165787,"apply.run_cpu_time_us":4845,"apply.run_wall_time_us":133223,"num_ops":11,"prepare.queue_time_us":305645,"prepare.run_cpu_time_us":28493,"prepare.run_wall_time_us":967015,"raft.queue_time_us":172228,"raft.run_cpu_time_us":48464,"raft.run_wall_time_us":234327,"replication_time_us":1462063,"thread_start_us":768485,"threads_started":4,"wal-append.queue_time_us":357168,"wal-append.run_cpu_time_us":45477,"wal-append.run_wall_time_us":2202607}]]}
I20260709 12:17:24.460988 19042 raft_consensus.cc:2368] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:24.410885 18373 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:24.425360 20578 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.421283 20565 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.460623 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.380631 20648 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.475087 18370 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:24.431734 20239 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.475787 18447 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.394490 20682 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:24.418128 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74908.7us
I20260709 12:17:24.460438 20667 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:24.502872 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:24.479496 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.415391 19035 rpcz_store.cc:276] 0709 12:17:23.337561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.743470 (+405909us) service_pool.cc:224] Handling call
0709 12:17:24.363411 (+619941us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54006202d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.363581 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439592320}
I20260709 12:17:24.482336 20454 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.487113 18373 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:24.513052 18373 raft_consensus.cc:614] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:24.493018 20722 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 12:17:24.460495 20563 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:24.504032 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43521.3us
I20260709 12:17:24.505683 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.491562 20703 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.516168 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:24.486739 20650 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:24.520841 20724 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:17:24.523412 20697 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:24.523989 20726 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:17:24.524288 20737 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:17:24.438683 20537 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:24.435667 20706 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.499338 20322 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.529090 20509 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:24.556011 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 201033us
I20260709 12:17:24.464360 20732 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.513623 20707 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.513965 20565 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:24.552623 20695 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.529280 20563 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.510317 20667 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.496296 20643 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.538432 20454 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.538664 20703 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.497264 20688 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:17:24.572876 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 93703.3us
W20260709 12:17:24.573668 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43798.8us
I20260709 12:17:24.641251 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:24.642668 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.575117 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.643595 18374 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:17:24.643989 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 68987.8us
W20260709 12:17:24.585652 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69577.4us
I20260709 12:17:24.645798 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.646443 18374 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 2.
I20260709 12:17:24.585043 20509 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:24.644250 20239 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:24.649343 20740 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.649505 20239 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.649973 20729 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.125s
W20260709 12:17:24.651441 20727 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.380s	user 0.001s	sys 0.001s
I20260709 12:17:24.651758 20239 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:24.652016 20723 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:17:24.503500 20735 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:17:24.593711 20707 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:24.662461 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:24.664475 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 491) took 1570 ms (client timeout 1864 ms). Trace:
W20260709 12:17:24.665027 19042 rpcz_store.cc:269] 0709 12:17:23.093732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.094065 (+   333us) service_pool.cc:224] Handling call
0709 12:17:24.664240 (+1570175us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400466754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:24.664402 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2877052800}
I20260709 12:17:24.668180 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:24.669847 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.544332 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:24.622710 20356 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:17:24.662359 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.676410 18374 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.671996 18291 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:24.672257 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:24.672677 20609 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.680238 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:24.673601 20742 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:17:24.674825 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.686604 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.627413 20667 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.627637 20563 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.685017 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.673321 20729 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:24.659713 18447 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:24.688256 19042 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:24.687409 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:24.634954 20565 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:24.627851 20739 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.658267 20509 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.706830 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.708128 18293 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.689045 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:24.690382 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 146142us
I20260709 12:17:24.639186 20703 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.680094 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:17:24.777071 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102350us
I20260709 12:17:24.693514 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:24.780453 19040 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:24.675637 20707 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:24.722163 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 105417us
I20260709 12:17:24.580120 20551 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.708727 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:24.675442 20688 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:24.565739 20647 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:24.843195 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 196031us
I20260709 12:17:24.706704 20207 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.769340 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.776176 18300 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:17:24.775554 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86610.8us
I20260709 12:17:24.786513 18375 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:24.848047 18297 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:24.692183 20570 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.708033 20696 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.708611 20735 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:24.836615 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58147.2us
W20260709 12:17:24.776238 20284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.040s	sys 0.071s
W20260709 12:17:24.839017 20584 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.002s
W20260709 12:17:24.841158 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 132505us
I20260709 12:17:24.841995 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:24.773464 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:24.710271 20740 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.690586 20622 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:17:24.852605 18448 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:24.695462 20239 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:17:24.623301 20585 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:24.835812 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 149291us
I20260709 12:17:24.884730 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.846520 20609 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:24.853330 20284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.040s	sys 0.071s
I20260709 12:17:24.747453 20560 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.855726 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.895666 18449 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:24.856658 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.859923 18445 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.844262 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:24.897751 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53593us
I20260709 12:17:24.898469 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:24.863945 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:24.900427 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 215479us
I20260709 12:17:24.894048 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:24.865687 20717 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.140s	user 0.002s	sys 0.003s
I20260709 12:17:24.866197 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.866421 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:24.836004 20347 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:24.875512 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:24.888007 19036 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:24.880136 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:24.772935 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:24.852636 20735 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.864887 18374 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:24.926987 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.928566 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.859834 20207 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:24.841830 20193 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.687s	user 0.005s	sys 0.005s
W20260709 12:17:24.904336 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 130529us
I20260709 12:17:24.942564 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:24.907819 18297 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:24.846316 20679 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:24.879711 20733 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.846930 20695 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.847100 20688 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:24.914619 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.915951 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:24.920274 20585 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.918589 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:24.964177 18370 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:24.964272 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:24.852453 20616 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:17:24.925179 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:24.846472 20739 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:17:24.897471 20697 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.876652 20674 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.893146 20728 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.870334 20584 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.946719 20732 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.944284 20560 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:17:24.948416 20564 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.947s	user 0.006s	sys 0.009s
I20260709 12:17:24.923801 20757 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.996793 20757 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:24.951494 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:24.998680 18297 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:24.954819 18412 leader_election.cc:400] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:24.999531 18412 leader_election.cc:403] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:24.951577 20755 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.955454 20643 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:24.956324 20743 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.941599 20452 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:17:24.917469 18369 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:25.015622 18369 pending_rounds.cc:85] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 3
I20260709 12:17:24.889834 20648 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:17:24.911475 20682 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:24.919749 20650 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:24.943702 20649 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:24.965831 18293 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.952638 20733 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.957808 20695 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.983057 20688 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:24.937250 20735 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.935560 20284 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.986781 18373 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:25.057179 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 172577us
I20260709 12:17:24.945668 20193 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:25.025861 20657 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.730s	user 0.007s	sys 0.002s
W20260709 12:17:24.993221 20557 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.000s
W20260709 12:17:24.994663 20644 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.822s	user 0.008s	sys 0.000s
I20260709 12:17:24.994895 20728 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:24.996012 20564 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:24.919210 20609 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 12:17:25.005352 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.990497 20707 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.011272 20643 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:25.012125 18291 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.958434 18375 raft_consensus.cc:2376] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6. Re-sending same reply.
I20260709 12:17:24.935704 20661 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:17:25.061486 20649 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:25.088786 20705 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.923s	user 0.003s	sys 0.012s
I20260709 12:17:24.902452 20565 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.024278 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:25.030968 18441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 480) took 1699 ms (client timeout 1904 ms). Trace:
I20260709 12:17:25.110657 18297 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:25.110598 18441 rpcz_store.cc:269] 0709 12:17:23.331980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.364772 (+ 32792us) service_pool.cc:224] Handling call
0709 12:17:25.030787 (+1666015us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002a9c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.030910 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371250048}
I20260709 12:17:25.012117 20743 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:25.058351 18373 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:24.987840 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 118677us
I20260709 12:17:25.059576 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:25.056499 20733 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.985082 20563 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:25.034969 20521 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
W20260709 12:17:25.080034 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 216177us
I20260709 12:17:25.082408 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:25.083267 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:25.083966 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:25.084772 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:25.088824 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 186212us
I20260709 12:17:25.088302 18443 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
W20260709 12:17:25.010033 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 64704.6us
I20260709 12:17:25.098217 18293 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:25.171459 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:25.098640 18445 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:25.172363 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:25.010944 20207 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.103926 20621 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.830s	user 0.005s	sys 0.005s
I20260709 12:17:25.012506 20560 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:25.104236 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:25.178133 18446 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:25.073572 20695 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.106102 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:25.170032 20452 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.022352 18369 pending_rounds.cc:107] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:25.182098 18369 raft_consensus.cc:2889] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:25.073136 20284 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.088162 20469 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:25.105739 20753 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:17:25.103387 20644 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.101809 20570 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.107820 20649 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:25.170469 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:25.170194 20563 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.168414 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:25.208259 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:25.088027 20657 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.099846 20688 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:17:25.073777 20759 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 12:17:25.176698 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:25.124990 20729 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.105190 20728 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.119001 20743 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:25.232056 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 172580us
I20260709 12:17:25.232219 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:25.196743 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 112065us
I20260709 12:17:25.104341 20757 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.202567 20584 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.211221 18449 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:25.240069 18300 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:25.240815 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:25.241366 18300 raft_consensus.cc:2376] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
I20260709 12:17:25.177573 20643 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:25.196697 20616 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:25.224941 18296 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:25.191581 20284 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:25.207983 20704 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.244s	user 0.011s	sys 0.002s
W20260709 12:17:25.173626 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 87336.4us
I20260709 12:17:25.172876 18293 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:25.174182 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:25.257264 18375 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 2.
I20260709 12:17:25.172647 20760 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:25.173616 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:25.261871 18412 leader_election.cc:403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:17:25.213963 20709 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.181s	user 0.002s	sys 0.011s
W20260709 12:17:25.232273 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55692.9us
I20260709 12:17:25.263680 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:25.207391 20732 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.233727 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:25.210817 20570 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.266734 18369 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:25.242033 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:25.528097 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:25.234591 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:25.233448 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:25.196554 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:25.531002 18300 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:25.272053 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 98506.8us
W20260709 12:17:25.534107 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.606s (SENT) [suppressed 14 similar messages]
I20260709 12:17:25.211982 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:25.559180 20755 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:25.559569 19035 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:25.492799 20735 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.173553 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:25.561939 18297 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:25.561900 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 388311us
W20260709 12:17:25.425022 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 247424us
W20260709 12:17:25.417682 20386 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.002s
I20260709 12:17:25.604601 20386 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:25.395449 20537 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.000s
I20260709 12:17:25.356488 20564 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.440622 20570 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.688751 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 375) took 1386 ms. Trace:
I20260709 12:17:25.259793 20239 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.695739 20239 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.697680 20537 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.699450 19042 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:25.700639 20537 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.443603 20616 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.703346 20616 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:25.707517 20207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.007s	sys 0.026s
W20260709 12:17:25.708133 20207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.007s	sys 0.026s
I20260709 12:17:25.708372 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 383) took 1013 ms. Trace:
I20260709 12:17:25.201856 20452 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.329093 20454 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:17:25.200167 20469 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.332721 20471 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:25.427966 20641 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:25.398321 20644 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:25.408512 20578 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.908s	user 0.005s	sys 0.006s
I20260709 12:17:25.244113 20609 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:25.192118 20621 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:25.800119 20284 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.004s
I20260709 12:17:25.358346 20643 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.800606 20284 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.801170 19036 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.856s	user 0.014s	sys 0.004s
I20260709 12:17:25.801725 19036 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:25.802078 19040 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:25.802445 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 503) took 1140 ms. Trace:
I20260709 12:17:25.264364 20657 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:25.803517 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:25.300384 20673 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.128s	user 0.011s	sys 0.000s
I20260709 12:17:25.804808 20673 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.806383 20790 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:25.328969 20681 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.473344 20696 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:25.811472 20696 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:25.812162 20681 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.813105 20696 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.814491 20696 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:25.814805 20681 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:25.489713 20703 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.004s
I20260709 12:17:25.815774 18372 raft_consensus.cc:2368] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:25.815923 20703 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.815796 20609 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.270769 20709 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.819548 20657 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.820403 20657 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.821792 20709 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.822376 18298 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:25.439713 20713 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.127s	user 0.012s	sys 0.000s
I20260709 12:17:25.818878 20703 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:25.261583 20704 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:25.824852 20704 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:25.368721 20705 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:25.348527 20707 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.858433 20707 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:25.859055 20707 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.859647 20667 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.152s	user 0.006s	sys 0.014s
I20260709 12:17:25.859841 20673 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.402024 20728 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.232642 20729 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.869656 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1574) took 1014 ms. Trace:
I20260709 12:17:25.870195 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1565) took 1778 ms. Trace:
I20260709 12:17:25.449893 20733 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.266425 20732 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:25.354295 20736 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 12:17:25.412057 20738 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.947s	user 0.011s	sys 0.008s
W20260709 12:17:25.321803 20748 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
W20260709 12:17:25.345140 20752 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
W20260709 12:17:25.345654 20758 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:17:25.252617 20766 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:17:25.285743 20762 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:25.231796 20761 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:17:25.332693 20764 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:25.252544 20772 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:25.352705 20769 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:25.321193 20770 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:25.361483 20771 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:17:25.497478 20743 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.564152 20735 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.564652 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 378) took 1249 ms. Trace:
W20260709 12:17:25.565332 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 331828us
I20260709 12:17:25.565816 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:25.573720 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:25.592164 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:25.592422 20585 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:25.610559 18293 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:25.611665 20755 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.614307 20386 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.577064 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:25.484920 20670 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.129s	user 0.013s	sys 0.000s
I20260709 12:17:25.611675 18370 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:25.678009 20649 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.692822 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1561) took 1011 ms. Trace:
I20260709 12:17:25.708935 18297 rpcz_store.cc:276] 0709 12:17:24.700335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.213636 (+513301us) service_pool.cc:224] Handling call
0709 12:17:25.708082 (+494446us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b540040c754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.708283 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71578496}
I20260709 12:17:25.716180 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:25.699245 20239 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:25.732849 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 198970us
I20260709 12:17:25.689416 18299 rpcz_store.cc:276] 0709 12:17:24.301930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.312160 (+ 10230us) service_pool.cc:224] Handling call
0709 12:17:24.312416 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.436730 (+124314us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:24.464634 (+ 27904us) log.cc:844] Serialized 14 byte log entry
0709 12:17:24.620626 (+155992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:24.620697 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:24.620761 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.373231 (+752470us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b180047f080. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:25.688033 (+314802us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001cd9d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:25.688149 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.688482 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107577,"prepare.queue_time_us":34650,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":6767,"raft.queue_time_us":101235,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":17228,"spinlock_wait_cycles":300815104,"thread_start_us":54424,"threads_started":1,"wal-append.queue_time_us":153350}
W20260709 12:17:25.771536 20509 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.920s	user 0.013s	sys 0.006s
I20260709 12:17:25.780900 20641 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:25.791970 19039 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:25.792541 20467 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.794869 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1565) took 1092 ms. Trace:
I20260709 12:17:25.797505 20578 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:25.476181 20697 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:25.448669 20706 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:25.802840 19036 rpcz_store.cc:276] 0709 12:17:24.661969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.675026 (+ 13057us) service_pool.cc:224] Handling call
0709 12:17:25.802317 (+1127291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:25.825006 20657 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:25.824463 20648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.021s	sys 0.016s
I20260709 12:17:25.836347 20705 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:25.846271 20616 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.847589 18367 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:25.800601 20621 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:25.851686 20555 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.852080 20284 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:25.855460 20695 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.008s
I20260709 12:17:25.855669 20452 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:25.856014 20609 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.825619 20704 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:25.823400 20713 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.866464 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 77077.1us
I20260709 12:17:25.869858 20790 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.353984 20731 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:17:25.870119 18443 rpcz_store.cc:276] 0709 12:17:24.854958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.856352 (+  1394us) service_pool.cc:224] Handling call
0709 12:17:25.869228 (+1012876us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54002030d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.869425 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440251136}
I20260709 12:17:25.870610 20729 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:25.870826 18444 rpcz_store.cc:276] 0709 12:17:24.091256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.283844 (+192588us) service_pool.cc:224] Handling call
0709 12:17:24.284130 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.413292 (+129162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.413325 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:25.747719 (+1334394us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:25.764379 (+ 16660us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.792036 (+ 27657us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:25.792114 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.792981 (+   867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.869736 (+ 76755us) raft_consensus.cc:1685] finished
0709 12:17:25.869754 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.870022 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163803,"raft.queue_time_us":117643,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":15101,"spinlock_wait_cycles":20745472,"thread_start_us":23757,"threads_started":1,"wal-append.queue_time_us":96845}
I20260709 12:17:25.893132 18294 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:25.860250 20667 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:25.887415 18441 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:26.045825 20732 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.046219 20704 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.048113 20386 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.048684 20697 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.050814 20386 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.052383 20755 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:26.053411 18300 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:26.053861 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1963) took 1185 ms. Trace:
I20260709 12:17:26.054203 18300 rpcz_store.cc:276] 0709 12:17:24.868383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.532350 (+663967us) service_pool.cc:224] Handling call
0709 12:17:26.053807 (+521457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.054909 20794 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.056084 20551 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.058491 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:26.058859 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:26.059921 20670 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.060271 20657 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.060717 20657 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.060665 20670 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.064951 20667 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:26.061221 20657 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.066977 20705 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.068392 20705 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.069861 20695 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:26.071357 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 108091us
I20260709 12:17:26.072680 20285 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.074414 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1566) took 1384 ms. Trace:
W20260709 12:17:26.052500 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 495846us
I20260709 12:17:26.055171 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:26.039220 18296 rpcz_store.cc:276] 0709 12:17:24.315536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.316424 (+   888us) service_pool.cc:224] Handling call
0709 12:17:25.564208 (+1247784us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54001cc0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.564448 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086145792}
I20260709 12:17:26.074823 18294 rpcz_store.cc:276] 0709 12:17:24.689542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.532506 (+842964us) service_pool.cc:224] Handling call
0709 12:17:25.892060 (+359554us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b5800003698. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:25.892421 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.039136 (+146715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.074342 (+ 35206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3719936}
I20260709 12:17:26.039388 18291 rpcz_store.cc:276] 0709 12:17:24.681006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.681177 (+   171us) service_pool.cc:224] Handling call
0709 12:17:24.681515 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.102526 (+421011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.692131 (+589605us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:25.692577 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759848320}
I20260709 12:17:26.089260 20585 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.091741 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 1957) took 1226 ms. Trace:
I20260709 12:17:26.092146 18293 rpcz_store.cc:276] 0709 12:17:24.865072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.172155 (+307083us) service_pool.cc:224] Handling call
0709 12:17:26.086236 (+914081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.093174 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:26.093295 20743 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.093911 18293 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:26.094693 19039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 640) took 2788 ms (client timeout 2081 ms). Trace:
I20260709 12:17:26.055789 20794 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:25.946707 18372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1486) took 1641 ms (client timeout 1796 ms). Trace:
I20260709 12:17:26.095983 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:26.094980 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:26.017745 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 781737us
I20260709 12:17:26.106215 18300 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
W20260709 12:17:26.105176 18372 rpcz_store.cc:269] 0709 12:17:24.308475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.311904 (+  3429us) service_pool.cc:224] Handling call
0709 12:17:25.946403 (+1634499us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54004150d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.946601 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3436922624}
I20260709 12:17:25.901872 20707 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.107896 18299 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:26.102316 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1563) took 1419 ms. Trace:
W20260709 12:17:26.074810 20648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.025s	sys 0.021s
I20260709 12:17:26.098428 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 490) took 1903 ms. Trace:
I20260709 12:17:26.097316 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:26.107203 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1500) took 1181 ms. Trace:
I20260709 12:17:26.099392 20771 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:26.114732 20771 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:26.115056 18294 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:26.116207 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:26.117143 20695 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.118309 18299 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:26.119503 18375 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:26.097751 20555 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.119823 18299 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:26.120453 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:25.910131 18298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 376) took 1597 ms (client timeout 2097 ms). Trace:
I20260709 12:17:26.110551 18369 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.945755 20647 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:26.120944 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 544179us
I20260709 12:17:25.932960 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1625) took 1231 ms. Trace:
W20260709 12:17:26.095156 19039 rpcz_store.cc:269] 0709 12:17:23.306413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.309650 (+  3237us) service_pool.cc:224] Handling call
0709 12:17:26.094442 (+2784792us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b540020bcd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:26.094590 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5509366016}
I20260709 12:17:26.121050 18375 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
W20260709 12:17:26.122387 18298 rpcz_store.cc:269] 0709 12:17:24.312370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.316483 (+  4113us) service_pool.cc:224] Handling call
0709 12:17:25.909822 (+1593339us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54002751d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:25.910035 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3263599488}
I20260709 12:17:26.053682 20755 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.102258 20322 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.041036 18292 rpcz_store.cc:276] 0709 12:17:24.689203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.689325 (+   122us) service_pool.cc:224] Handling call
0709 12:17:24.689602 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.985441 (+295839us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:24.990829 (+  5388us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.174528 (+183699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.174578 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:25.174617 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.465699 (+291082us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18000f2260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:25.764478 (+298779us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54001cb6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:25.766682 (+  2204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.781529 (+ 14847us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65696,"prepare.queue_time_us":5641,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":87398,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":20266,"spinlock_wait_cycles":687680640,"thread_start_us":117629,"threads_started":1,"wal-append.queue_time_us":226428}
I20260709 12:17:26.112574 18370 rpcz_store.cc:276] 0709 12:17:24.925658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.964972 (+ 39314us) service_pool.cc:224] Handling call
0709 12:17:24.965214 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.047185 (+1081971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.103282 (+ 56097us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b5400114c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:26.107104 (+  3822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473464192}
I20260709 12:17:26.042833 20673 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.052413 20759 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:26.102846 20469 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:26.056954 20551 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.125205 20670 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.112231 18442 rpcz_store.cc:276] 0709 12:17:24.194876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.285709 (+ 90833us) service_pool.cc:224] Handling call
0709 12:17:24.285987 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.342816 (+ 56829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.342861 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.343447 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.555270 (+211823us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:24.555312 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.555389 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.098014 (+1542625us) raft_consensus.cc:1685] finished
0709 12:17:26.098057 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.098251 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108186,"raft.queue_time_us":57501,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":17505,"thread_start_us":211439,"threads_started":1,"wal-append.queue_time_us":492616}
I20260709 12:17:26.111287 18295 rpcz_store.cc:276] 0709 12:17:24.682518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.682633 (+   115us) service_pool.cc:224] Handling call
0709 12:17:24.682921 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.683234 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.683292 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.689891 (+  6599us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.893916 (+204025us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:24.893949 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.894008 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.101897 (+1207889us) raft_consensus.cc:1685] finished
0709 12:17:26.101942 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.102114 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"raft.queue_time_us":474,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11124,"thread_start_us":192362,"threads_started":1,"wal-append.queue_time_us":297337}
I20260709 12:17:26.127612 20729 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.097281 20509 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.163060 18299 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:26.140479 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:26.161691 18367 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:26.167981 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:26.171622 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.173082 18441 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:26.173956 20285 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.174448 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.175902 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.175541 20641 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.176542 18443 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.177132 20641 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.179744 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.180472 18445 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:26.181690 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.183032 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.171490 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.192291 20322 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.194615 20772 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:26.195791 20772 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.161758 20555 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.192191 20673 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.205441 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:26.211861 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:26.160521 20609 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:26.225172 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.225920 20609 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.227550 20759 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.228327 20759 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.229267 20609 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:26.230886 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 81688.1us
I20260709 12:17:26.232770 20555 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.232863 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:26.233320 20555 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:26.234434 20673 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.234478 20555 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.235682 20516 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.236164 20516 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.236541 20516 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.236662 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1487) took 1927 ms. Trace:
I20260709 12:17:26.237186 18366 rpcz_store.cc:276] 0709 12:17:24.309010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.465682 (+156672us) service_pool.cc:224] Handling call
0709 12:17:24.497467 (+ 31785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.500812 (+  3345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.500843 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.501274 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.767584 (+266310us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:24.767648 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.767732 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.236261 (+1468529us) raft_consensus.cc:1685] finished
0709 12:17:26.236295 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.236491 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2976,"raft.queue_time_us":5643,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":21962,"thread_start_us":210017,"threads_started":1,"wal-append.queue_time_us":411738}
I20260709 12:17:26.241139 20509 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.243347 20762 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.245862 20762 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.245656 20516 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.246600 20701 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:26.247347 20701 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.131414 18368 rpcz_store.cc:276] 0709 12:17:24.701083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.707135 (+  6052us) service_pool.cc:224] Handling call
0709 12:17:24.843091 (+135956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.922990 (+ 79899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.936648 (+ 13658us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.946572 (+  9924us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.819834 (+873262us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b18003c70c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:17:25.836362 (+ 16528us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:25.836468 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.836569 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.918130 (+ 81561us) raft_consensus.cc:1685] finished
0709 12:17:25.918153 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.932739 (+ 14586us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78833,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12782,"spinlock_wait_cycles":15164544,"thread_start_us":52648,"threads_started":2,"wal-append.queue_time_us":953783}
I20260709 12:17:26.249163 18368 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:26.249778 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 410) took 1158 ms. Trace:
I20260709 12:17:26.247881 20516 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.251314 18371 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:26.251438 20732 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.253659 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1475) took 1956 ms. Trace:
I20260709 12:17:26.254027 20732 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.254113 18371 rpcz_store.cc:276] 0709 12:17:24.297709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.300877 (+  3168us) service_pool.cc:224] Handling call
0709 12:17:24.301157 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.252036 (+1950879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.253252 (+  1216us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b5400394c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:26.253561 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4146540800}
I20260709 12:17:26.255457 20733 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.258750 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.266449 18440 raft_consensus.cc:2410] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:26.261289 18291 raft_consensus.cc:2410] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:26.262768 18294 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 3: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:26.250038 20701 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:26.203470 20707 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:26.247017 20467 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.279539 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.279697 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.280282 18441 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:26.280875 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.281154 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.282053 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.282235 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.282677 18445 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 2.
I20260709 12:17:26.282902 18441 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:26.283613 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.283731 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.284701 20609 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.285321 20609 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.263918 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:26.244161 20616 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.204782 20771 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.284727 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.427263 18443 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:26.545256 20673 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:26.644863 18441 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:26.325661 20579 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.645366 20673 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.319104 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 411) took 1227 ms. Trace:
I20260709 12:17:26.645783 20579 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.260331 20611 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:26.646865 20611 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:26.330295 20713 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.647750 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.648140 20609 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:26.405191 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52727us
I20260709 12:17:26.325876 18374 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:26.650015 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.324971 20705 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:26.650985 18440 raft_consensus.cc:2410] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:26.651163 20705 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.331278 20707 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.652585 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.653551 20705 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.333859 20729 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.654551 20790 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.655002 19035 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:26.332954 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 189859us
I20260709 12:17:26.329684 20681 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:26.423095 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 219894us
I20260709 12:17:26.326220 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 409) took 1234 ms. Trace:
I20260709 12:17:26.544629 20706 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.663332 18366 rpcz_store.cc:276] 0709 12:17:25.091529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.239001 (+1147472us) service_pool.cc:224] Handling call
0709 12:17:26.302810 (+ 63809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.325766 (+ 22956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.325834 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.325915 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:26.325959 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.326017 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.326139 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.401741 20657 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.250092 18368 rpcz_store.cc:276] 0709 12:17:25.091808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.248483 (+1156675us) service_pool.cc:224] Handling call
0709 12:17:26.248911 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.249532 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.249711 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.646394 18371 rpcz_store.cc:276] 0709 12:17:25.092040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.263595 (+1171555us) service_pool.cc:224] Handling call
0709 12:17:26.318353 (+ 54758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.318622 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.318654 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.318728 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:26.318783 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.318857 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.319013 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.681551 20771 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.681335 20706 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.330067 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:26.231015 20759 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.667964 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:26.632778 20798 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 12:17:26.656471 20729 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.682406 20681 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:26.688719 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:26.258776 20193 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:26.690574 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:26.697535 18369 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.002s
I20260709 12:17:26.631935 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:26.699349 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:26.653908 20707 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.332772 20516 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:26.732980 18296 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.025s	sys 0.055s
I20260709 12:17:26.647746 20673 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:26.736469 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:26.592162 20794 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.422253 20762 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.324440 20649 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:26.648173 20611 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:26.764423 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:26.880800 18372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.005s	sys 0.012s
W20260709 12:17:27.163058 18372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.013s	sys 0.023s
W20260709 12:17:26.912941 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.162973 18366 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:27.161218 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 948143us
W20260709 12:17:26.765260 18370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.031s	sys 0.022s
W20260709 12:17:26.837783 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54868.7us
W20260709 12:17:27.164846 18370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.041s	sys 0.050s
I20260709 12:17:27.161669 18369 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:26.984402 18297 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:27.166160 18369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1630) took 1903 ms (client timeout 2000 ms). Trace:
I20260709 12:17:26.966300 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 392) took 1135 ms. Trace:
I20260709 12:17:26.885250 18443 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.167043 18299 rpcz_store.cc:276] 0709 12:17:25.838749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.120575 (+281826us) service_pool.cc:224] Handling call
0709 12:17:26.120881 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.163542 (+ 42661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.965962 (+802420us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b540040d654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:26.966217 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86316416}
W20260709 12:17:27.166795 18369 rpcz_store.cc:269] 0709 12:17:25.262250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.715512 (+453262us) service_pool.cc:224] Handling call
0709 12:17:27.166087 (+1450575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.162285 18445 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:26.684141 20609 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:27.171427 18296 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.031s	sys 0.090s
I20260709 12:17:27.181124 18441 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:26.764998 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 2.079s (SENT)
W20260709 12:17:26.967736 18374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1499) took 2042 ms (client timeout 2079 ms). Trace:
W20260709 12:17:26.837064 20755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.008s	sys 0.027s
W20260709 12:17:27.186045 18374 rpcz_store.cc:269] 0709 12:17:24.925411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.928301 (+  2890us) service_pool.cc:224] Handling call
0709 12:17:26.929557 (+2001256us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003f2354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:26.967659 (+ 38102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2922979712}
I20260709 12:17:27.199925 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:26.899524 20509 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:26.903509 20551 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:26.799503 20565 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:27.204854 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.224169 18446 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.226110 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.227818 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.227110 19038 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:26.887964 20704 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.880945 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:27.197430 20755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.032s	sys 0.041s
W20260709 12:17:27.233506 18367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1491) took 2923 ms (client timeout 1815 ms). Trace:
W20260709 12:17:27.233898 18367 rpcz_store.cc:269] 0709 12:17:24.312794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.315370 (+  2576us) service_pool.cc:224] Handling call
0709 12:17:27.233292 (+2917922us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54000c99d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.233408 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3472399616}
I20260709 12:17:27.206794 20609 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:26.899712 20732 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.308s
I20260709 12:17:27.201448 20707 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.235890 20707 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.237771 20609 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:27.238034 20707 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:27.238945 20551 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.239768 20551 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.242535 20551 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.211506 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1503) took 2285 ms. Trace:
I20260709 12:17:27.200969 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:27.189198 20647 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.048s	user 0.013s	sys 0.000s
I20260709 12:17:27.169334 20616 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:27.243893 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51116us
I20260709 12:17:27.244287 20733 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.244413 20616 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.246361 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.247277 20616 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.193660 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.248533 18368 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:27.248445 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.249143 18294 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:27.249547 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.249904 20785 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.250137 18368 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.250464 20785 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.214308 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:27.216374 18441 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:27.216435 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 517637us
I20260709 12:17:27.219619 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.252719 20322 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.223392 20794 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.253405 20794 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.243115 18375 rpcz_store.cc:276] 0709 12:17:24.926332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.155033 (+1228701us) service_pool.cc:224] Handling call
0709 12:17:26.204855 (+ 49822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.243457 (+ 38602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.243489 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.244000 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:17:26.660374 (+416374us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:26.660417 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.660487 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.211304 (+550817us) raft_consensus.cc:1685] finished
0709 12:17:27.211325 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.211446 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27838,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":416022,"threads_started":1,"wal-append.queue_time_us":956890}
I20260709 12:17:27.254545 20647 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:27.255573 20794 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.234931 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.255884 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 412) took 2163 ms. Trace:
I20260709 12:17:27.211372 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:27.213251 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2010) took 1042 ms. Trace:
I20260709 12:17:26.765089 20193 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:27.247722 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.249001 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.216445 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.252875 19042 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:27.243441 18373 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:27.257282 18373 pending_rounds.cc:85] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 1
W20260709 12:17:27.215920 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.731781 20709 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:27.244854 20733 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.256224 18375 rpcz_store.cc:276] 0709 12:17:25.092278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.254940 (+2162662us) service_pool.cc:224] Handling call
0709 12:17:27.255266 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.255474 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.255498 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.255563 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:27.255646 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.255701 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.255805 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:27.196816 20697 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.147s	user 0.009s	sys 0.016s
W20260709 12:17:27.263952 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50577.6us
I20260709 12:17:27.256614 18448 rpcz_store.cc:276] 0709 12:17:26.171119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.209655 (+ 38536us) service_pool.cc:224] Handling call
0709 12:17:27.213186 (+1003531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.264621 18446 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:27.265877 20644 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:27.235291 20643 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:27.269089 20706 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:27.270009 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.270701 20733 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.181838 20516 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.271541 20709 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.272246 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.271989 20193 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.272760 18373 pending_rounds.cc:107] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:27.272779 20762 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.213508 20657 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:27.276712 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:27.364310 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.019s	sys 0.079s
W20260709 12:17:27.348835 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 155828us
I20260709 12:17:27.302116 18369 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.362871 18373 raft_consensus.cc:2889] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:27.295240 19039 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.000s
I20260709 12:17:27.368116 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.367383 19039 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:27.368757 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.214454 20611 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:27.368662 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:27.371387 18374 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:27.381425 19038 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:27.384166 19040 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.320613 20579 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.323819 20585 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.364235 20681 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.392728 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 394) took 1518 ms. Trace:
W20260709 12:17:27.395465 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.392328 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:27.399586 18447 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.009s	sys 0.018s
I20260709 12:17:27.306785 20609 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
W20260709 12:17:27.401320 18447 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.010s	sys 0.019s
I20260709 12:17:27.399653 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.315562 20704 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.414674 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:27.421926 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 53436.2us
I20260709 12:17:27.373396 20697 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.424410 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.394171 20706 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.430445 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:27.423300 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.405931 20644 raft_consensus.cc:2764] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:17:27.425731 20771 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.000s
W20260709 12:17:27.426024 20555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.006s	sys 0.026s
I20260709 12:17:27.443872 18367 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.425202 20585 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:27.448247 18292 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:27.447057 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:27.449548 18299 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:27.451362 18444 raft_consensus.cc:2410] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:27.428093 20193 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.454599 18371 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.427212 20647 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.457034 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.457718 18300 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:27.452917 20555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.007s	sys 0.026s
I20260709 12:17:27.427826 20609 consensus_queue.cc:1243] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:27.419448 20579 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:27.456295 20585 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:27.413733 20643 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.466311 20732 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:27.486958 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.488480 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.487696 18300 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.491418 18298 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.457360 20565 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.494536 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:27.445575 20322 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:27.501434 20322 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.508790 20643 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.492151 20732 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.509590 20732 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.511876 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:27.515115 20509 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:27.515522 20509 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.515906 20509 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.516702 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1951) took 2654 ms. Trace:
I20260709 12:17:27.517091 18297 rpcz_store.cc:276] 0709 12:17:24.861939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.133496 (+1271557us) service_pool.cc:224] Handling call
0709 12:17:26.984177 (+850681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.174158 (+189981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.516488 (+342330us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400090450. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.516639 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847757184}
I20260709 12:17:27.518427 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.519137 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:27.519562 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.518603 18440 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:27.487674 20647 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.522722 20697 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.523479 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:27.524260 20647 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:27.525063 18443 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:27.525298 20804 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:17:27.525482 20802 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.526098 20802 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.526925 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1505) took 2530 ms. Trace:
I20260709 12:17:27.527338 18370 rpcz_store.cc:276] 0709 12:17:24.996575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.202470 (+1205895us) service_pool.cc:224] Handling call
0709 12:17:26.220213 (+ 17743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.224418 (+  4205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.224462 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.225139 (+   677us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.165630 (+940491us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.165682 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.165765 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.526727 (+360962us) raft_consensus.cc:1685] finished
0709 12:17:27.526746 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.526873 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7265,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":203,"thread_start_us":940045,"threads_started":1,"wal-append.queue_time_us":1159893}
I20260709 12:17:27.529399 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.531446 18374 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:27.531117 20565 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.533485 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.534795 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.525113 20616 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:17:27.538702 18292 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:27.541443 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 62322us
I20260709 12:17:27.542600 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:27.475659 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.545802 18374 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:27.540619 20673 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 12:17:27.557480 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63073.8us
I20260709 12:17:27.490077 20585 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:27.562662 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 74958us
I20260709 12:17:27.578934 19040 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:27.583911 18369 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:27.491484 20551 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.593456 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.467015 20706 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.603103 20762 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.604246 18292 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:27.604769 20322 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.605350 20706 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.605899 20709 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:27.606254 20805 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:17:27.608153 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.608363 20516 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.609530 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:27.610008 18444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2002) took 1471 ms (client timeout 1606 ms). Trace:
W20260709 12:17:27.610589 18444 rpcz_store.cc:269] 0709 12:17:26.168187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.171317 (+  3130us) service_pool.cc:224] Handling call
0709 12:17:27.609711 (+1438394us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400376254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.609904 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940991872}
W20260709 12:17:27.589362 20806 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.358s	user 0.004s	sys 0.000s
I20260709 12:17:27.610999 20771 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:27.611512 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4.
I20260709 12:17:27.613170 20467 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:27.615533 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:27.651678 18446 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:27.616250 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.394104 18295 rpcz_store.cc:276] 0709 12:17:25.874301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.164435 (+290134us) service_pool.cc:224] Handling call
0709 12:17:26.199715 (+ 35280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.210412 (+ 10697us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:17:26.836807 (+626395us) log.cc:844] Serialized 14 byte log entry
0709 12:17:27.392370 (+555563us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.392428 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:17:27.392491 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.392565 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.392678 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1116171,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":9602,"thread_start_us":953238,"threads_started":2}
I20260709 12:17:27.616946 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.617663 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1502) took 2691 ms. Trace:
I20260709 12:17:27.619143 18370 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:27.619823 20704 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.621065 20802 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.622131 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:27.599138 20810 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:17:27.626947 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.668534 18292 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:27.628867 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.630847 20785 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:27.594657 20681 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:27.589210 20794 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.592392 20551 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:27.595198 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:27.634100 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.643060 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 738) took 1314 ms. Trace:
I20260709 12:17:27.676916 19040 rpcz_store.cc:276] 0709 12:17:26.328621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.329705 (+  1084us) service_pool.cc:224] Handling call
0709 12:17:27.642606 (+1312901us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54006bd854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.642973 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576179072}
I20260709 12:17:27.643671 18366 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:27.649426 18443 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:27.658373 18367 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:27.651168 20467 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:27.662963 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51667.3us
I20260709 12:17:27.663098 19042 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.664134 18368 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:27.641739 20709 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.664381 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.653705 18373 rpcz_store.cc:276] 0709 12:17:24.926100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.115948 (+189848us) service_pool.cc:224] Handling call
0709 12:17:27.090181 (+1974233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.378476 (+288295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.617333 (+238857us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54003f2350. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.617546 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4231403776}
I20260709 12:17:27.666797 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:27.667420 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.674952 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.704288 18370 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:27.687022 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1632) took 1091 ms. Trace:
I20260709 12:17:27.691475 18375 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:27.677850 20681 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.705070 18366 rpcz_store.cc:276] 0709 12:17:26.595650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.685543 (+ 89893us) service_pool.cc:224] Handling call
0709 12:17:27.686938 (+1001395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.674192 20707 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.707823 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 414) took 2615 ms. Trace:
I20260709 12:17:27.708083 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.651465 20565 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.708212 18375 rpcz_store.cc:276] 0709 12:17:25.092743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.264341 (+2171598us) service_pool.cc:224] Handling call
0709 12:17:27.686912 (+422571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.705240 (+ 18328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.707619 (+  2379us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540034ab50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:27.707743 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864846592}
I20260709 12:17:27.707422 20707 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.700953 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:27.707823 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 120798us
I20260709 12:17:27.712941 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:27.714779 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.678107 20551 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.715562 18375 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:27.717163 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 415) took 2617 ms. Trace:
I20260709 12:17:27.717497 18375 rpcz_store.cc:276] 0709 12:17:25.099469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.716269 (+2616800us) service_pool.cc:224] Handling call
0709 12:17:27.716653 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.716836 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.716861 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.716925 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:27.716966 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.717024 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.717115 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.671411 20643 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:27.677959 20771 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:27.724567 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:27.703495 20706 raft_consensus.cc:2764] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:27.709046 20681 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:27.702697 20644 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:27.730160 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.739348 18367 raft_consensus.cc:2410] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:27.733896 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:27.727077 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.735926 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:27.744391 18445 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:17:27.722894 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.748610 18369 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:27.749565 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1510) took 2041 ms. Trace:
I20260709 12:17:27.731942 18368 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:27.751274 18369 rpcz_store.cc:276] 0709 12:17:25.708269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.747960 (+2039691us) service_pool.cc:224] Handling call
0709 12:17:27.748319 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.749210 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.749485 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.751960 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.754457 18440 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:27.755090 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1504) took 2801 ms. Trace:
I20260709 12:17:27.726082 20643 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.711472 20709 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.772826 20585 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.773521 20643 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.699507 20794 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.774952 18370 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.775712 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1509) took 2068 ms. Trace:
I20260709 12:17:27.776032 18370 rpcz_store.cc:276] 0709 12:17:25.707760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.745920 (+2038160us) service_pool.cc:224] Handling call
0709 12:17:27.774763 (+ 28843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.775461 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.775653 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.777092 20794 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.758276 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.759951 20516 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.763355 20814 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.784572 18374 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:27.764107 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.785468 18441 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:27.786945 18375 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:27.765134 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:27.767444 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:27.750586 20609 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:17:27.770224 20467 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.773622 20709 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:27.779114 18448 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:27.756456 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 43886.6us
I20260709 12:17:27.782614 18371 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:27.787732 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1506) took 2529 ms. Trace:
I20260709 12:17:27.792791 18441 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:27.813977 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.815138 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1631) took 1240 ms. Trace:
I20260709 12:17:27.755462 18372 rpcz_store.cc:276] 0709 12:17:24.953288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.246242 (+1292954us) service_pool.cc:224] Handling call
0709 12:17:26.246707 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.259413 (+ 12706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.259447 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.259957 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.163823 (+903866us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:27.163946 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.164050 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.754898 (+590848us) raft_consensus.cc:1685] finished
0709 12:17:27.754917 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.755035 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19718,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":28715,"thread_start_us":903442,"threads_started":1,"wal-append.queue_time_us":1151094}
I20260709 12:17:27.815685 18371 rpcz_store.cc:276] 0709 12:17:26.575064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.699450 (+124386us) service_pool.cc:224] Handling call
0709 12:17:27.423142 (+723692us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b5800083a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57eee 000000000036eb39 00007f3c6ddcb1e0 00007f3c6de940c4 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:27.815062 (+391920us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16696704}
I20260709 12:17:27.817716 18371 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:27.824925 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.826016 18300 raft_consensus.cc:2410] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:27.828991 18375 rpcz_store.cc:276] 0709 12:17:25.257968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.718727 (+2460759us) service_pool.cc:224] Handling call
0709 12:17:27.778556 (+ 59829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.787438 (+  8882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.787668 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.831554 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:27.832474 18443 raft_consensus.cc:2368] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:27.833710 18371 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:27.738580 20785 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 12:17:27.820480 20705 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.010s	sys 0.018s
I20260709 12:17:27.744167 20644 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.852074 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1635) took 1051 ms. Trace:
I20260709 12:17:27.800326 20555 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:27.801092 20643 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:27.814762 19036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.010s	sys 0.010s
I20260709 12:17:27.857326 18369 rpcz_store.cc:276] 0709 12:17:26.800657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.779059 (+978402us) service_pool.cc:224] Handling call
0709 12:17:27.789669 (+ 10610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.843397 (+ 53728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.843425 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.851792 (+  8367us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:27.851842 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.851900 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.852022 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.859346 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.819535 20565 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.865979 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.831197 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1507) took 2565 ms. Trace:
W20260709 12:17:27.852103 20772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.019s	sys 0.028s
I20260709 12:17:27.868806 18300 raft_consensus.cc:2376] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4. Re-sending same reply.
W20260709 12:17:27.851593 20705 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.010s	sys 0.019s
I20260709 12:17:27.849582 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:27.876760 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.871555 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.871850 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:27.877645 18297 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:27.877758 18293 raft_consensus.cc:2376] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
I20260709 12:17:27.764024 20771 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.794234 20516 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.872666 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.852736 20467 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:27.865515 19036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.011s	sys 0.011s
W20260709 12:17:27.874688 20772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.020s	sys 0.028s
I20260709 12:17:27.889968 18298 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:27.864070 20555 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.894814 18292 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:27.895426 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50059us
W20260709 12:17:27.898473 19009 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.002s	sys 0.021s
W20260709 12:17:27.900300 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 364589us
W20260709 12:17:27.900666 19009 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.003s	sys 0.021s
W20260709 12:17:27.900952 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 532957us
W20260709 12:17:27.896493 18296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.006s	sys 0.026s
W20260709 12:17:27.902114 18296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.006s	sys 0.026s
W20260709 12:17:27.892136 18449 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.013s	sys 0.009s
I20260709 12:17:27.899174 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:27.902791 18449 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.013s	sys 0.009s
I20260709 12:17:27.895972 18299 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:27.908754 18293 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:27.848311 20729 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:17:27.912542 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1567) took 3217 ms. Trace:
I20260709 12:17:27.914618 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.891393 20585 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:27.900159 20555 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:27.875572 20643 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:27.876807 20565 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:27.907421 20657 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:27.872668 18368 rpcz_store.cc:276] 0709 12:17:25.258279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.721281 (+2463002us) service_pool.cc:224] Handling call
0709 12:17:27.721731 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.780037 (+ 58306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.823737 (+ 43700us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.891822 20705 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:17:27.913556 20704 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.944362 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.951627 18299 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:27.948819 18375 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:17:27.955582 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48621us
I20260709 12:17:27.955799 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.956769 18298 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:27.950570 18295 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:27.949375 20729 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:17:27.953512 20611 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.001s
I20260709 12:17:27.975587 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:27.946324 20772 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:27.963776 18447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.006s	sys 0.011s
W20260709 12:17:27.992386 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 449814us
W20260709 12:17:28.001583 18447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.006s	sys 0.011s
I20260709 12:17:28.002430 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:27.993755 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 338502us
W20260709 12:17:28.003986 20812 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:17:27.955742 20657 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:28.006789 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.954001 20704 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:28.047595 20704 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.049026 20704 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.955412 20707 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.055886 20609 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.056384 20609 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.057538 20609 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.027055 18293 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:17:28.036954 20732 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.004s
I20260709 12:17:28.037688 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.038380 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.039098 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.065095 18366 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.065052 18375 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:28.031450 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 513) took 1233 ms. Trace:
I20260709 12:17:28.041589 20673 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:28.070187 20673 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.070616 20673 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.071712 18445 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.073089 18366 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:28.072703 20657 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.041932 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.074872 18374 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:27.950279 20785 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:17:28.042635 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.075487 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.043135 20826 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:28.039875 20509 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.004s
I20260709 12:17:28.067054 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.104079 18298 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:28.044157 20585 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:27.952622 20817 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:17:28.048019 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:27.948899 20764 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 12:17:28.039528 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.124668 20509 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.039768 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.136240 18368 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:28.104172 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.059067 20609 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:28.137581 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.137727 20509 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:28.138118 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.138612 20826 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.139782 18372 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.140936 20826 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.048020 20322 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:28.149618 20193 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.001s
I20260709 12:17:28.149849 20322 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:28.151015 20322 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.152277 20322 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.124444 20516 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.156034 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.156589 18441 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:28.157953 20565 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.162681 18448 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:28.106134 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.164155 18369 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:27.917160 18296 rpcz_store.cc:276] 0709 12:17:24.694723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.095717 (+1400994us) service_pool.cc:224] Handling call
0709 12:17:26.096162 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.096514 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:27.181034 (+1084520us) log.cc:844] Serialized 14 byte log entry
0709 12:17:27.912061 (+731027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.912108 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:27.912180 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.912313 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.912454 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1239278,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":55,"raft.queue_time_us":18125,"spinlock_wait_cycles":1920,"thread_start_us":1765475,"threads_started":2}
I20260709 12:17:28.127840 20764 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.127614 20681 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.065831 18447 rpcz_store.cc:276] 0709 12:17:26.797784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.797913 (+   129us) service_pool.cc:224] Handling call
0709 12:17:26.798280 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.798598 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:27.413670 (+615072us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.029795 (+616125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:28.029866 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:28.029934 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.030706 (+   772us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.030863 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":918938,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":40750,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":1162700,"threads_started":2}
I20260709 12:17:28.129055 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.129691 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.097491 20732 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:28.171730 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 305554us
W20260709 12:17:28.174661 18415 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.796s (SENT)
I20260709 12:17:28.169243 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 512) took 1372 ms. Trace:
I20260709 12:17:28.133667 20551 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:28.091130 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59066.4us
I20260709 12:17:28.147877 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:28.148316 20816 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:28.148391 20649 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:28.166320 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118258us
I20260709 12:17:28.195655 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.064594 20772 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.198498 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:28.168272 18294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.002s	sys 0.021s
W20260709 12:17:28.199402 18294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.002s	sys 0.021s
I20260709 12:17:28.186043 18372 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:28.189416 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:28.168226 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.172065 20733 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.790s	user 0.003s	sys 0.011s
I20260709 12:17:28.196944 18448 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:28.171622 20647 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:28.205324 18369 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:28.172238 20565 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.177661 18449 rpcz_store.cc:276] 0709 12:17:26.797295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.813662 (+ 16367us) service_pool.cc:224] Handling call
0709 12:17:26.906631 (+ 92969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.960450 (+ 53819us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:27.905180 (+944730us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.168905 (+263725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:28.168962 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:28.169015 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.169078 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.169184 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":754444,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":41,"raft.queue_time_us":1424,"thread_start_us":981927,"threads_started":2}
I20260709 12:17:28.166261 20764 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:28.210604 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:28.211927 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.187593 20826 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.215570 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.210754 20733 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.216979 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:28.200475 20649 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:28.208755 20729 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.221136 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.234294 18371 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:28.210587 20509 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:28.224920 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50261.5us
I20260709 12:17:28.225921 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.240728 18375 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:28.226948 18370 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.170430 20585 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.228477 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.230979 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:28.242146 18374 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
W20260709 12:17:28.232455 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 256608us
I20260709 12:17:28.213080 20709 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:28.242790 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.245862 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:28.247053 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 343420us
I20260709 12:17:28.252512 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.249251 20565 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.254417 18291 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:28.236796 20814 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:28.251507 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:28.270831 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.271876 18374 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:28.275527 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.276163 18369 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
W20260709 12:17:28.275476 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64354us
I20260709 12:17:28.245136 20709 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:28.229427 20826 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.248967 20817 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:28.289639 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.253983 20509 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:28.294332 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 380079us
I20260709 12:17:28.254944 20839 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.242317 20729 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.310797 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.275789 20814 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.245492 20697 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:17:28.318045 18374 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:17:28.320272 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 47414.7us
I20260709 12:17:28.291661 20817 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.323395 20814 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:28.316502 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 108008us
I20260709 12:17:28.323818 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 508) took 1672 ms. Trace:
I20260709 12:17:28.324352 20771 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:28.327486 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.326589 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.290158 20709 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:28.347437 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.329870 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1508) took 3068 ms. Trace:
I20260709 12:17:28.330797 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.348284 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 419) took 1081 ms. Trace:
W20260709 12:17:28.344372 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 92951us
I20260709 12:17:28.315701 20837 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.355433 20649 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.355719 20837 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:28.356014 20649 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.356135 20771 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.356667 20649 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.321257 20764 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.356736 20771 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.357697 20764 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.357638 20649 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.358107 20673 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.358932 20771 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.332867 18368 raft_consensus.cc:2376] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6. Re-sending same reply.
I20260709 12:17:28.331349 20846 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:28.361406 20846 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.362383 20846 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.334026 20817 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:28.354884 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.345131 19036 rpcz_store.cc:276] 0709 12:17:26.651172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.651341 (+   169us) service_pool.cc:224] Handling call
0709 12:17:26.651659 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.652461 (+   802us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:26.652810 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.255428 (+1602618us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.286536 (+ 31108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:28.288581 (+  2045us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:28.291546 (+  2965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.315378 (+ 23832us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.323752 (+  8374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4609,"prepare.queue_time_us":703,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":173094,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":5578,"spinlock_wait_cycles":1184000,"thread_start_us":1454094,"threads_started":3,"wal-append.queue_time_us":842473}
I20260709 12:17:28.364334 20585 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.360026 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.356163 20837 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:28.365410 20837 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.372483 18369 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:28.347836 18373 rpcz_store.cc:276] 0709 12:17:25.258563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.743759 (+2485196us) service_pool.cc:224] Handling call
0709 12:17:27.767868 (+ 24109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.768363 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.768403 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.769016 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.247780 (+478764us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:28.247831 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.247913 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.326712 (+ 78799us) raft_consensus.cc:1685] finished
0709 12:17:28.326731 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.326864 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9468,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":478297,"threads_started":1,"wal-append.queue_time_us":487545}
I20260709 12:17:28.373360 18375 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:28.376590 20732 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.377597 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.377836 20840 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:28.378410 20840 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:28.379418 20729 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.379323 20840 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.380606 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 421) took 1113 ms. Trace:
I20260709 12:17:28.380549 20834 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:28.381165 18370 rpcz_store.cc:276] 0709 12:17:27.267079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.814190 (+547111us) service_pool.cc:224] Handling call
0709 12:17:27.814516 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.241867 (+427351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.380366 (+138499us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54000c85d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:28.380553 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948051328}
I20260709 12:17:28.377444 20509 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:28.383679 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 106229us
I20260709 12:17:28.374222 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:28.387750 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.389400 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.349888 20755 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:28.353832 20839 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.329346 20555 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.334731 20657 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 12:17:28.396886 18415 leader_election.cc:336] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.815s (SENT)
I20260709 12:17:28.375936 20697 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:28.380589 20845 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 12:17:28.404047 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 149986us
W20260709 12:17:28.409065 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57885.4us
I20260709 12:17:28.405833 20681 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.418092 18445 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:28.421523 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.405426 20697 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.402978 20834 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.408510 20732 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:28.438299 20814 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:28.439364 20822 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:17:28.426673 20771 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:17:28.441674 20771 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.426441 20846 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:17:28.442173 20771 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.442580 20732 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:28.442334 20817 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.442965 20732 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.443874 20732 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.430466 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.407426 20551 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.431581 18264 leader_election.cc:400] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:28.428409 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.418226 20839 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.433149 20643 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.433317 20834 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.414113 20733 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.445169 20555 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.399252 20565 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.465040 20842 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.445086 20732 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.470046 20709 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.451530 18264 leader_election.cc:403] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:28.454279 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.418471 20755 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.424829 20729 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:28.458786 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.460592 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.462291 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.427057 20681 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.431663 20467 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.491700 19037 raft_consensus.cc:2368] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10.
I20260709 12:17:28.471221 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:28.491873 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64114.5us
W20260709 12:17:28.492026 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63753.5us
I20260709 12:17:28.492411 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.492998 18441 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.493054 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.471829 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.494083 19042 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:28.472434 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.473604 20644 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.475358 20647 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.503249 20647 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.505764 19044 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:28.506058 20516 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:28.506516 20516 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:28.507956 20772 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.499075 18373 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:28.495816 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.443115 20771 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.476372 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.515645 19039 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.477128 20697 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.478050 20847 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:17:28.478691 20814 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.522159 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.480068 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.413009 20826 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:28.481259 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.481412 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1569) took 1080 ms. Trace:
W20260709 12:17:28.490141 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 179421us
I20260709 12:17:28.490146 19041 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.495158 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.495441 20657 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.495442 20729 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.495986 20755 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.496361 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.496506 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:28.540920 18338 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.093s (SENT)
I20260709 12:17:28.498251 20551 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.498379 20834 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.498811 20733 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5.
I20260709 12:17:28.499428 20681 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.500769 20643 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:28.501169 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.501368 20846 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:28.502259 20673 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.521051 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.465984 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.524093 18291 rpcz_store.cc:276] 0709 12:17:27.400862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.404753 (+  3891us) service_pool.cc:224] Handling call
0709 12:17:27.421638 (+ 16885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.256569 (+834931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.477086 (+220517us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540004c4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:28.481358 (+  4272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1903482112}
I20260709 12:17:28.525228 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.523455 18369 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:28.599092 20697 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:28.619992 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.620633 18440 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.541208 18374 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:28.541467 18371 raft_consensus.cc:2376] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5. Re-sending same reply.
I20260709 12:17:28.543460 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.543515 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.621541 18373 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:28.595117 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.548082 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.595088 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.595482 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:28.622928 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 101965us
I20260709 12:17:28.595742 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.605000 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.595153 19036 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.626811 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:28.627810 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 161906us
I20260709 12:17:28.541532 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.631138 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.631841 18296 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:28.547610 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.549108 20771 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.539697 19035 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:28.615180 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.617108 20846 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.651181 20709 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.650283 20647 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.533116 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:28.652216 18293 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.618299 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.618592 18441 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:28.606089 20837 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.621958 19040 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:28.547948 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.624398 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:28.626727 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:28.596493 20551 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.664462 20681 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.658042 18444 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:28.619010 18368 raft_consensus.cc:2376] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4. Re-sending same reply.
I20260709 12:17:28.666244 20551 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.667258 18375 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:28.670372 18293 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:28.670598 20840 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:28.671720 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.672258 20826 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.635365 18295 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
W20260709 12:17:28.636051 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110906us
I20260709 12:17:28.635836 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.673799 20516 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.672260 20818 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.675261 20818 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:28.675694 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.676429 18297 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:28.676048 20818 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.677546 20818 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:28.678781 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 137891us
I20260709 12:17:28.541066 20852 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.638991 18446 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:28.677232 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.679978 18297 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:28.632839 20857 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:28.640153 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.646059 20657 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:28.647467 20729 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.651511 20846 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:28.618232 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.659420 18291 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:28.662642 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet ca383fa1243547c3b32dfbd6857a3fba: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:28.664209 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.671782 20859 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.628656 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:28.685768 18440 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:28.637916 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.695564 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.715839 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.716647 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:28.718137 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:28.718595 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:28.718958 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:28.719936 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.720618 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:28.721403 20647 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.634430 20649 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.726178 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.728818 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.731808 19043 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:28.852707 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:28.860502 18292 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:28.348850 18367 rpcz_store.cc:276] 0709 12:17:27.266492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.783528 (+517036us) service_pool.cc:224] Handling call
0709 12:17:27.820051 (+ 36523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.872950 (+ 52899us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:28.249746 (+376796us) log.cc:844] Serialized 14 byte log entry
0709 12:17:28.347666 (+ 97920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:28.347775 (+   109us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:28.347888 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.347974 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.348178 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11605,"thread_start_us":372606,"threads_started":2}
W20260709 12:17:28.677767 20858 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:17:28.862133 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.854235 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.862744 19037 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:28.863091 20785 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:28.860522 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 231952us
W20260709 12:17:28.758275 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.716107 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55657.8us
I20260709 12:17:28.865412 18444 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:17:28.867281 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.872519 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.872597 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:28.872905 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.867172 18447 raft_consensus.cc:2410] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:28.870782 18369 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:28.867990 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102624us
W20260709 12:17:28.874493 18339 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.635s (SENT)
W20260709 12:17:28.874470 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.874675 18292 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:28.875589 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.876068 20705 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.875919 20842 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:28.732966 20837 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.877228 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.733538 20839 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.879877 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.866896 20649 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.884757 19037 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.888940 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:28.891697 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.891821 20826 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.871788 20785 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.893014 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.879199 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:28.896529 18370 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:28.899008 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.899748 19041 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.892479 20826 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:28.871457 20565 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:17:28.873890 19036 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:28.855281 18368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0931a54c049145dfad672173c0b29a40"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:28.893067 20817 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:28.902598 20826 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.915577 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.916004 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.904217 19037 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:28.918035 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.905431 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.910183 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:28.918952 18372 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:28.904287 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.903846 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.904017 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.910437 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.922233 20817 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:28.923055 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.923604 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.924322 18370 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:28.917137 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.910477 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.930927 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.931917 19038 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:28.935948 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:28.932785 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 7d788f8182ca4b13bddd2acdf511873f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:28.938179 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:28.939404 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.928966 20859 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:28.906951 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.942063 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.922999 20852 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:28.943747 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.943635 20852 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.906865 20842 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.944496 20842 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:28.932160 20732 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:28.945514 20764 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:28.945369 20842 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.945830 20852 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:28.946133 20732 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:28.945366 20565 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:28.943806 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.947530 18373 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:28.947098 20732 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.907644 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.948271 19036 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.943621 20839 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.948325 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.949196 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.949388 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:28.949822 19036 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:28.938731 20467 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.950562 20467 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:28.950757 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.946839 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.906896 20785 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.907465 18368 raft_consensus.cc:606] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:28.943041 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.951460 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:28.929153 20817 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:28.952692 20467 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:28.923574 20697 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:28.938572 20845 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:17:28.936162 20673 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:28.955354 20516 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.932734 20551 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:28.941408 20555 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.950664 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.960325 18372 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:17:28.961802 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.987111 19037 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.942952 20814 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:28.996371 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:28.998499 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.999218 19038 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:29.000053 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:29.001076 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:29.002031 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:29.003440 20837 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.003598 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.981478 20322 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.004298 19040 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:29.004526 20322 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:29.005088 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:29.005431 20322 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:29.006204 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:29.006842 19035 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:28.986325 20551 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:29.007795 20551 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:29.015455 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64849.8us
I20260709 12:17:29.016237 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:28.997412 20814 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.017814 20814 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:29.020951 20817 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.023124 20509 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.023528 20509 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:29.023615 20704 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:28.972064 18368 raft_consensus.cc:614] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:29.024458 20509 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:28.992404 18444 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:28.982252 20697 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.026297 20697 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:29.028012 20509 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:29.032318 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.007728 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:29.044695 18370 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:29.044977 18374 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:29.024838 18373 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:28.972092 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:29.047003 18339 leader_election.cc:336] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.721s (SENT)
I20260709 12:17:28.976452 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:28.993091 20467 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.048822 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 691992us
W20260709 12:17:29.051556 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 79582.7us
I20260709 12:17:29.053673 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:29.054291 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 91389.1us
I20260709 12:17:29.003042 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:29.058050 19038 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:28.902791 20649 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.061225 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:29.065379 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:29.067862 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:29.060223 20467 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.075664 18375 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:29.077155 20467 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:29.083856 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:29.079150 19037 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:29.086122 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:29.079471 19038 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:29.089447 19038 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:29.090952 19038 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:29.081007 18339 leader_election.cc:336] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.828s (SENT)
I20260709 12:17:29.095425 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.047457 20845 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:17:29.097555 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:29.150820 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58672.6us
I20260709 12:17:29.060101 20755 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:29.157963 18339 leader_election.cc:336] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.902s (SENT)
I20260709 12:17:29.065287 20565 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.171407 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.172329 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.154939 20733 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.154510 20837 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:17:29.151010 20709 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:29.197503 18339 leader_election.cc:336] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.961s (SENT)
W20260709 12:17:29.162532 20832 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:17:29.151830 20467 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:29.201684 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 104215us
I20260709 12:17:29.176677 20755 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.186614 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.187345 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:29.216913 18448 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:29.187477 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.156945 20839 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:29.192086 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.078742 20814 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.168408 20872 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.198897 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.199950 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 135901us
I20260709 12:17:29.186242 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:29.229794 18446 raft_consensus.cc:2376] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
I20260709 12:17:29.202452 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.202895 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.211705 18370 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.212178 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.213295 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.213846 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:29.214623 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:29.232465 18292 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:29.167517 20673 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:29.215378 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.218225 18444 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:29.234158 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.234839 18444 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:29.235911 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.237151 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.238086 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.238984 18444 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:29.212889 18298 pending_rounds.cc:85] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 4
I20260709 12:17:29.239431 18298 pending_rounds.cc:107] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:29.239737 18298 raft_consensus.cc:2889] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.197952 20709 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:29.246802 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.248034 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.248669 18296 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:29.221527 18447 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:29.223251 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.226953 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.220218 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.228935 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:29.244588 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.244953 18366 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:29.245210 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 190073us
W20260709 12:17:29.253734 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 1d0efe89255942c4bd576ff4b96cd91d: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:29.252072 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:29.257951 19040 raft_consensus.cc:2368] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:29.258153 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.259670 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 62220us
I20260709 12:17:29.262466 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.263497 18446 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:29.267488 18299 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:29.268409 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.224436 20837 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:29.220996 20856 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:17:29.219422 20467 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.276989 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.761869 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 559439us
I20260709 12:17:29.759871 20842 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:29.756385 20551 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.000s
I20260709 12:17:29.764240 20551 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:29.616688 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 430818us
I20260709 12:17:29.766618 20551 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:29.762336 18446 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:17:29.314393 20832 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:17:29.769694 20657 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:29.770349 20657 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:29.770892 20837 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.771202 20657 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.772550 20657 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.773146 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.773990 20565 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.774832 18440 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:29.775233 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:29.775928 20467 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:29.776047 20565 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.367188 20876 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.776465 20565 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:29.347644 20873 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:17:29.777032 20565 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.777041 20876 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.778043 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1610) took 1096 ms. Trace:
I20260709 12:17:29.305871 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:29.778745 18447 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.255210 20673 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:29.778409 18441 rpcz_store.cc:276] 0709 12:17:28.681722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.681893 (+   171us) service_pool.cc:224] Handling call
0709 12:17:28.682300 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.682802 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.682841 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.862029 (+179188us) log.cc:844] Serialized 26 byte log entry
0709 12:17:29.157297 (+295268us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:29.157337 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.157401 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.777841 (+620440us) raft_consensus.cc:1685] finished
0709 12:17:29.777859 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.777978 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":387,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"thread_start_us":294846,"threads_started":1,"wal-append.queue_time_us":412482}
I20260709 12:17:29.320048 18292 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:29.554142 20882 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:17:29.781098 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.782197 20755 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:29.633057 19044 consensus_meta.cc:287] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.001s
I20260709 12:17:29.782945 19044 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:29.783418 20555 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:29.783912 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.784359 19035 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:29.784958 20755 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.786268 20876 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:29.783644 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 654) took 1105 ms. Trace:
I20260709 12:17:29.787035 19044 rpcz_store.cc:276] 0709 12:17:28.677987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.679026 (+  1039us) service_pool.cc:224] Handling call
0709 12:17:29.783590 (+1104564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:29.311975 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.641024 20839 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:29.788870 20755 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:29.790959 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:29.792481 20755 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.793942 18263 leader_election.cc:400] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
W20260709 12:17:29.763156 20785 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
I20260709 12:17:29.797004 20785 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:29.798254 18445 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:29.799685 20785 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:29.800504 18263 leader_election.cc:403] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:29.805253 20755 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:29.805399 18445 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:29.806516 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 529) took 1146 ms. Trace:
W20260709 12:17:29.798240 18194 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 292) took 12971 ms (client timeout 15000 ms). Trace:
I20260709 12:17:29.809250 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:29.810369 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:29.254973 20709 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.799265 20839 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:29.811934 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.730641 18373 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.003s
I20260709 12:17:29.260424 20826 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.814875 18373 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:29.306262 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.294090 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50063.4us
W20260709 12:17:29.426263 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.820246 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.820657 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.821156 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.806855 18445 rpcz_store.cc:276] 0709 12:17:28.660512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.716390 (+ 55878us) service_pool.cc:224] Handling call
0709 12:17:29.806361 (+1089971us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b5400258fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:29.806450 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2139764992}
I20260709 12:17:29.825764 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.763794 18448 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:29.828084 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.829195 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.662606 20847 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:29.830570 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.689343 20509 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.000s
W20260709 12:17:29.819087 20817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.010s
W20260709 12:17:29.715683 20516 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.003s
I20260709 12:17:29.447640 18197 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:29.864514 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.832898 18445 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:29.866446 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:29.868024 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.555115 20846 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.000s
W20260709 12:17:29.380995 20875 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:17:29.864907 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.866068 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.873986 18446 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:29.874866 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.884337 18448 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:29.824853 20551 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.872732 18442 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:29.673076 20814 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:29.815414 20733 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.834038 20785 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.877575 18292 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:29.817394 20876 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.878592 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:29.879021 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:29.830188 20704 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:29.870797 20817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.000s	sys 0.011s
W20260709 12:17:29.875020 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.018s	sys 0.038s
I20260709 12:17:29.833926 20771 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.913724 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.018s	sys 0.038s
I20260709 12:17:29.451911 20697 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:29.914022 18440 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.913864 18444 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:29.891656 19044 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:29.870028 20847 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:29.920971 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 668886us
I20260709 12:17:29.921289 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:29.977785 18298 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:29.932974 18197 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 3, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 3 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:29.985078 19037 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 6. Ops: []
W20260709 12:17:29.988902 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.914521 20704 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.913830 20876 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.917227 20551 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.025846 18300 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:30.026437 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:30.034868 18440 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:30.035754 18442 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:29.930281 20845 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.882905 20846 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.040588 18443 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:30.043154 18440 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:30.052474 18444 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:30.045513 18442 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:30.046973 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 519) took 1138 ms. Trace:
I20260709 12:17:30.048519 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:30.047142 19042 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:30.043166 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:30.053145 19037 rpcz_store.cc:276] 0709 12:17:28.908527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.087581 (+179054us) service_pool.cc:224] Handling call
0709 12:17:29.087952 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:29.986307 (+898355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.046770 (+ 60463us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54002166d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:30.046920 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964162304}
I20260709 12:17:30.072013 18197 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 4, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:30.060149 18369 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:30.027285 20704 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.931576 20771 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.049198 19035 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:30.048910 20755 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
W20260709 12:17:30.097769 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54679.1us
I20260709 12:17:30.095804 18294 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:30.092993 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.098901 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 407) took 1418 ms. Trace:
W20260709 12:17:30.099516 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 119571us
I20260709 12:17:30.051419 20509 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.037925 20876 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.110898 19044 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:30.111613 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:30.115815 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:30.116894 18373 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
W20260709 12:17:30.115984 18339 leader_election.cc:336] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.155s (SENT)
I20260709 12:17:30.109885 18295 rpcz_store.cc:276] 0709 12:17:28.680048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.862948 (+182900us) service_pool.cc:224] Handling call
0709 12:17:28.863268 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:28.863656 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.863682 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:28.872777 (+  9095us) log.cc:844] Serialized 52 byte log entry
0709 12:17:29.155249 (+282472us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:29.275140 (+119891us) log.cc:844] Serialized 14 byte log entry
0709 12:17:29.762042 (+486902us) log.cc:844] Serialized 14 byte log entry
0709 12:17:29.762521 (+   479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.762611 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.098683 (+336072us) raft_consensus.cc:1685] finished
0709 12:17:30.098704 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.098846 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5032,"prepare.queue_time_us":182911,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":2994,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":20498,"thread_start_us":352879,"threads_started":2,"wal-append.queue_time_us":342630}
I20260709 12:17:30.117245 18373 pending_rounds.cc:85] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:17:30.117985 18373 pending_rounds.cc:107] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:17:30.118256 18373 raft_consensus.cc:2889] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:30.092810 20891 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:30.116091 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.132252 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:30.132899 18197 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:17:29.808421 18194 rpcz_store.cc:269] 0709 12:17:16.827153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.827900 (+   747us) service_pool.cc:224] Handling call
0709 12:17:29.797867 (+12969967us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:24.685303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:24.940001 (+254698us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:24.944168 (+  4167us) write_op.cc:435] Acquired schema lock
0709 12:17:24.952217 (+  8049us) tablet.cc:662] Decoding operations
0709 12:17:25.192566 (+240349us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:25.198807 (+  6241us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:25.208116 (+  9309us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:17:25.354607 (+146491us) tablet.cc:701] Row locks acquired
0709 12:17:25.354624 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:17:25.354813 (+   189us) write_op.cc:270] Start()
0709 12:17:25.354939 (+   126us) write_op.cc:276] Timestamp: P: 1783599445354744 usec, L: 0
0709 12:17:25.354961 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:17:25.394933 (+ 39972us) log.cc:844] Serialized 3238 byte log entry
0709 12:17:26.369188 (+974255us) op_driver.cc:464] REPLICATION: finished
0709 12:17:27.437628 (+1068440us) write_op.cc:301] APPLY: starting
0709 12:17:27.454321 (+ 16693us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:27.527914 (+ 73593us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:27.528082 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:27.529658 (+  1576us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:27.530104 (+   446us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:17:27.530168 (+    64us) write_op.cc:312] APPLY: finished
0709 12:17:27.530690 (+   522us) log.cc:844] Serialized 65 byte log entry
0709 12:17:27.995292 (+464602us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:28.098379 (+103087us) write_op.cc:454] Released schema lock
0709 12:17:28.124935 (+ 26556us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:28.137464 (+ 12529us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:28.137495 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2970717,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":1031522,"apply.run_cpu_time_us":22982,"apply.run_wall_time_us":734319,"num_ops":8,"prepare.queue_time_us":220022,"prepare.run_cpu_time_us":22789,"prepare.run_wall_time_us":1203742,"raft.queue_time_us":522858,"raft.run_cpu_time_us":64519,"raft.run_wall_time_us":411219,"replication_time_us":1013960,"thread_start_us":1629656,"threads_started":5,"wal-append.queue_time_us":1162287,"wal-append.run_cpu_time_us":75099,"wal-append.run_wall_time_us":2401294}]]}
I20260709 12:17:30.087592 20845 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.133904 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:30.197657 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65425.9us
W20260709 12:17:30.197350 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45693.1us
I20260709 12:17:30.206357 18197 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 1 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:30.134140 20891 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:30.216882 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:30.218395 20898 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.218962 20901 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:30.219512 20901 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:30.219404 20772 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:30.219735 20840 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:30.220623 20901 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.164036 20322 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:30.223124 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 2cbd14f2572940d797f0297b94365af4: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:17:30.226687 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 662) took 1323 ms (client timeout 1728 ms). Trace:
W20260709 12:17:30.227190 19042 rpcz_store.cc:269] 0709 12:17:28.903684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.903788 (+   104us) service_pool.cc:224] Handling call
0709 12:17:30.226483 (+1322695us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b5400660154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:30.226620 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208023808}
I20260709 12:17:30.222417 20772 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:30.228569 20772 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:30.228178 20890 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
W20260709 12:17:30.229715 20897 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:17:30.228865 20892 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:17:30.229888 18197 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:30.230897 18368 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:30.231289 20772 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.234010 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1585) took 1041 ms. Trace:
I20260709 12:17:30.234490 18294 rpcz_store.cc:276] 0709 12:17:29.192389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.193169 (+   780us) service_pool.cc:224] Handling call
0709 12:17:30.233788 (+1040619us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540009ef54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:30.233951 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1941825280}
W20260709 12:17:30.235728 20895 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:17:30.171126 20826 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.129959 20818 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:17:30.244623 18444 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:17:30.209285 20875 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:17:30.263334 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.265995 18368 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:30.281541 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 385831us
I20260709 12:17:30.279932 20551 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:30.134472 20509 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.197008 20673 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:30.285876 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 85444.6us
W20260709 12:17:30.291599 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.294587 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:30.291404 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 196980us
I20260709 12:17:30.113844 20847 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:30.303809 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 194410us
I20260709 12:17:30.304561 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:30.239424 20891 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.260368 20898 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.312027 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:30.309341 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.309600 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:30.283048 20733 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.281212 20322 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:30.290895 20872 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.323957 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:30.215267 20555 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:17:30.283601 20888 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:30.333276 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 514430us
I20260709 12:17:30.321820 20673 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.282316 20826 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.332309 20847 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:30.358387 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 139398us
I20260709 12:17:30.333894 20697 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:30.364050 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.329120 20516 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:17:30.368180 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:30.333456 20900 raft_consensus.cc:2764] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:30.350644 20846 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:30.326896 20733 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.384886 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:30.389012 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:30.396819 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7472368249998453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
W20260709 12:17:30.397454 18338 connection.cc:256] RPC call timeout handler was delayed by -1.724736824999809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 12:17:30.397977 18338 connection.cc:256] RPC call timeout handler was delayed by -1.5546368249997613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 12:17:30.398377 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3899368249999497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
W20260709 12:17:30.398728 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3431368249998741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 12:17:30.400413 20905 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.402402 20911 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:30.409071 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2027368249998744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:17:30.329694 20898 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.363063 20899 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:30.380017 20817 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.346669 20888 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.383540 20771 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:30.389833 20611 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.404502 20772 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.405140 20872 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.419898 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:30.746011 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 418994us
I20260709 12:17:30.787499 19044 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:30.645426 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.777349 19042 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:30.419878 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:30.449054 18447 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.670s	user 0.010s	sys 0.000s
W20260709 12:17:30.789930 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 370116us
W20260709 12:17:30.419034 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.790720 18447 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:30.790989 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:30.701318 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 364155us
I20260709 12:17:30.791420 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2040) took 1544 ms. Trace:
I20260709 12:17:30.389863 20910 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:30.426941 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.789803 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 477819us
I20260709 12:17:30.811108 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:30.812170 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:30.796576 18447 rpcz_store.cc:276] 0709 12:17:29.246565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.263465 (+ 16900us) service_pool.cc:224] Handling call
0709 12:17:30.791352 (+1527887us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:30.790161 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55542.9us
W20260709 12:17:30.814695 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.757978 20657 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.985s	user 0.009s	sys 0.004s
W20260709 12:17:30.736160 20908 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 12:17:30.424059 20896 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:17:30.796012 20899 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.479389 20872 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.558732 20846 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.483392 20697 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:30.814349 20551 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:30.795495 20901 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:30.507741 20914 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.551162 20785 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:30.823735 20837 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.481392 20611 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.796097 20900 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.783339 20509 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:30.808820 20839 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.424013 20755 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.480049 20842 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.348s
I20260709 12:17:30.835433 20826 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:30.735872 20876 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 12:17:30.780885 20913 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:30.468221 20817 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.420257 20912 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.456s
I20260709 12:17:30.839524 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:30.827881 20872 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:30.838529 18339 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.029s
I20260709 12:17:30.839915 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:30.462428 20322 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:30.856350 18339 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.007s	sys 0.030s
I20260709 12:17:30.856695 20785 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.859491 20929 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:30.860576 20657 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:30.847874 20839 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.835739 20911 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:17:30.832252 20733 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:30.464649 20904 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.436s
I20260709 12:17:30.423784 20905 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.851569 20551 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:30.863564 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:30.849480 20846 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:30.853263 20565 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.075s	user 0.010s	sys 0.002s
I20260709 12:17:30.874972 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:30.821682 20910 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.872877 18373 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:30.872222 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1616) took 1507 ms. Trace:
W20260709 12:17:30.858556 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 588650us
W20260709 12:17:30.878078 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.572s (SENT)
I20260709 12:17:30.876951 18445 rpcz_store.cc:276] 0709 12:17:29.364711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.882595 (+517884us) service_pool.cc:224] Handling call
0709 12:17:29.882914 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.887176 (+  4262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.887203 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.912729 (+ 25526us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.239590 (+326861us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:30.239635 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.240103 (+   468us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.872020 (+631917us) raft_consensus.cc:1685] finished
0709 12:17:30.872041 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.872161 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":164579,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"thread_start_us":240401,"threads_started":1,"wal-append.queue_time_us":423075}
W20260709 12:17:30.878613 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.675s (SENT) [suppressed 47 similar messages]
I20260709 12:17:30.880883 18368 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:17:30.880199 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64985.5us
I20260709 12:17:30.882210 18447 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:30.843011 20837 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.842792 20901 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:30.869029 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:30.893349 20657 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:30.895766 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:30.873615 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 62932.7us
I20260709 12:17:30.897260 18449 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:30.898759 20926 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:30.898838 20923 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:17:30.870478 20929 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.882598 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:30.867359 20888 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:30.852592 20876 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.841213 20917 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:17:30.833258 20899 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.838730 20914 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:30.854485 20913 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:31.044898 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:31.172192 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:31.047896 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 169290us
I20260709 12:17:31.038777 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 684) took 1145 ms. Trace:
W20260709 12:17:31.206756 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 324257us
W20260709 12:17:31.171875 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 288025us
I20260709 12:17:31.227352 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:30.899506 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.230808 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:31.242877 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:31.244396 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:31.245026 18449 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:31.245174 18441 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:30.899642 18443 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:31.016053 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:31.138351 20872 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:31.309854 18449 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:31.247256 18368 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:31.244714 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:31.218735 20898 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:31.152664 20817 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:30.994695 20907 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:17:31.172127 20926 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:31.105650 20839 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:31.331293 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 100545us
I20260709 12:17:31.333251 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:31.239666 19040 rpcz_store.cc:276] 0709 12:17:29.893325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.893434 (+   109us) service_pool.cc:224] Handling call
0709 12:17:30.301273 (+407839us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:30.301529 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.334666 (+ 33137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.334712 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.358129 (+ 23417us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.784956 (+426827us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:30.785041 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.785141 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.038479 (+253338us) raft_consensus.cc:1685] finished
0709 12:17:31.038511 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.038718 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32614,"raft.queue_time_us":35677,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":143,"spinlock_wait_cycles":23240576,"thread_start_us":410499,"threads_started":1,"wal-append.queue_time_us":629233}
I20260709 12:17:31.255900 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:31.244758 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:31.333326 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.248113 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.214883 20931 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.047185 20772 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.000s
W20260709 12:17:31.335409 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 90775.3us
W20260709 12:17:31.041040 20709 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.217s	user 0.008s	sys 0.004s
I20260709 12:17:30.886101 20565 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:31.030970 20673 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.000s
I20260709 12:17:31.217926 20913 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:31.343698 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.352741 18443 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:31.252182 20551 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:31.353108 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 108404us
I20260709 12:17:31.353297 18446 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:17:31.009505 20930 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 12:17:31.353802 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97977.5us
I20260709 12:17:31.354413 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:31.354638 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:31.026798 20846 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:31.030267 20905 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:31.023551 20942 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.332s
I20260709 12:17:31.493654 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:31.024240 20921 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.192s	user 0.003s	sys 0.000s
I20260709 12:17:31.354466 20729 raft_consensus.cc:2764] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:31.340950 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:31.030395 20611 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:31.355204 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:31.340090 20945 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.042160 20937 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:17:31.251021 20918 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.606s	user 0.002s	sys 0.000s
I20260709 12:17:31.363672 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:31.359789 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:31.036924 20704 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.908s	user 0.017s	sys 0.001s
I20260709 12:17:31.218894 20910 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:31.250635 20900 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:31.023726 20939 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.044153 20889 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.029142 20837 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:31.252126 20914 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:31.541913 18298 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:31.535960 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1594) took 1603 ms. Trace:
I20260709 12:17:31.502594 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:31.155485 20901 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:31.151829 20916 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.159730 20814 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:31.138904 20902 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:31.369818 18413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.009s	sys 0.017s
I20260709 12:17:31.416911 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:31.336911 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 497449us
I20260709 12:17:31.432494 18338 leader_election.cc:400] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10.
I20260709 12:17:31.023986 20912 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.349s
I20260709 12:17:31.493211 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:31.008661 20943 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:31.021402 20906 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.421s
I20260709 12:17:31.252648 20924 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:31.251955 20936 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:31.493227 18300 raft_consensus.cc:2368] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:17:30.994714 20920 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.668s
I20260709 12:17:31.501778 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:31.371340 20891 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.036s	user 0.011s	sys 0.001s
I20260709 12:17:31.650149 19040 raft_consensus.cc:2376] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5. Re-sending same reply.
W20260709 12:17:31.541819 18412 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.012s	sys 0.110s
W20260709 12:17:31.650945 18412 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.013s	sys 0.120s
W20260709 12:17:31.651245 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 684401us
I20260709 12:17:31.399894 20839 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.544786 20872 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:31.395927 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.355243 20941 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:31.579537 18291 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:31.586628 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.581073 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:31.541673 20937 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:31.364125 20847 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.989s	user 0.001s	sys 0.009s
I20260709 12:17:31.380816 18297 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:31.386090 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:31.581727 18299 rpcz_store.cc:276] 0709 12:17:29.932591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.934711 (+  2120us) service_pool.cc:224] Handling call
0709 12:17:29.999966 (+ 65255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.039279 (+ 39313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.039324 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.089562 (+ 50238us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.418096 (+328534us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.418170 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.418271 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.535679 (+1117408us) raft_consensus.cc:1685] finished
0709 12:17:31.535710 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.535892 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173311,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":42047,"thread_start_us":287452,"threads_started":1,"wal-append.queue_time_us":373831}
I20260709 12:17:31.635802 18338 leader_election.cc:403] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:17:31.636108 18413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.035s	sys 0.064s
W20260709 12:17:31.620797 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 60565.5us
I20260709 12:17:31.636701 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:31.637322 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 144157us
I20260709 12:17:31.581110 18296 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:17:31.646425 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 229635us
I20260709 12:17:31.442729 20673 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.339663 20818 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.355702 20949 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.581807 19042 raft_consensus.cc:2376] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5. Re-sending same reply.
I20260709 12:17:31.657969 18443 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:31.437196 20709 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.533877 20772 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.589954 20814 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:31.426177 20931 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.584084 20946 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.413s
I20260709 12:17:31.645817 20902 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:31.586280 20889 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.584218 20939 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.561939 20945 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:31.669188 19043 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:31.938548 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1590) took 2665 ms. Trace:
W20260709 12:17:31.729321 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 224466us
I20260709 12:17:31.369602 20898 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:31.977589 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.976207 18442 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:31.929669 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.977069 18293 rpcz_store.cc:276] 0709 12:17:29.264276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.819370 (+555094us) service_pool.cc:224] Handling call
0709 12:17:29.871030 (+ 51660us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:30.251881 (+380851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.251926 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:30.293920 (+ 41994us) log.cc:844] Serialized 52 byte log entry
0709 12:17:30.532048 (+238128us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:31.138355 (+606307us) log.cc:844] Serialized 14 byte log entry
0709 12:17:31.138694 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.138820 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.760218 (+621398us) raft_consensus.cc:1685] finished
0709 12:17:31.760247 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.929595 (+169348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33511,"prepare.queue_time_us":1030213,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":312565,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":107674,"thread_start_us":1004802,"threads_started":3,"wal-append.queue_time_us":509069}
W20260709 12:17:31.742830 18415 connection.cc:256] RPC call timeout handler was delayed by -1.1822692259997893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260709 12:17:31.544883 20944 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.648594 20565 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.001403 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:31.745476 20837 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.993268 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 356630us
I20260709 12:17:32.000974 18292 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 12:17:31.995332 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 623296us
I20260709 12:17:31.834075 20950 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.745555 20953 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.778296 20948 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 12:17:31.931936 20516 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:31.976213 20818 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.582388 20657 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.745754 20952 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:31.729369 20876 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:32.013265 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:31.980280 20898 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.988328 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:31.638036 20916 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.700599 20901 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:31.994805 20959 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:31.997792 20872 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10.
I20260709 12:17:31.998704 20944 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.985500 20956 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.084916 20944 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.085112 20956 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.020402 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:31.633882 20704 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.088019 20956 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:31.998704 18415 leader_election.cc:336] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.824s (SENT)
I20260709 12:17:31.667615 20891 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.088797 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90120us
I20260709 12:17:32.089109 18372 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:32.089453 18415 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.947s (SENT)
I20260709 12:17:32.089906 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:32.090097 20891 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:31.986428 20772 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.074988 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 86737.6us
I20260709 12:17:31.662961 20936 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:31.655468 20900 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.089193 20704 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:32.091029 18415 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.980s (SENT)
W20260709 12:17:32.071514 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 58368.5us
I20260709 12:17:32.094796 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:31.680691 20839 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.082914 20898 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.039333 20913 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:32.098337 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78009.9us
I20260709 12:17:32.103962 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:32.104388 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:32.067726 20903 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.015674 20837 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.037240 20847 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:17:32.113330 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:32.066079 20818 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.073838 20963 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.103677 20901 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:32.081988 20941 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.090883 20673 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:32.065418 20905 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 12:17:32.132392 18415 leader_election.cc:336] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.985s (SENT)
I20260709 12:17:32.097023 20920 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.990801 20949 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.065353 20516 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.091138 20924 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:32.144433 18339 leader_election.cc:336] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.045s (SENT)
I20260709 12:17:32.103513 20914 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.017017 20953 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.157096 18263 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.728s (SENT)
I20260709 12:17:32.018863 20950 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.098523 20899 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.138900 20906 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.130326 20872 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:32.080132 20952 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.184134 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:32.183082 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 78764.9us
I20260709 12:17:32.140687 20900 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.158269 20704 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:32.071168 20814 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.067785 20937 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.138252 20912 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:17:32.159492 20941 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.138182 20891 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:17:32.176822 20876 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.222275 20950 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.222095 20876 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.223423 20709 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.223731 20889 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.224170 20876 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:32.225067 20961 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:17:32.225630 20952 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.140540 20903 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.200956 20704 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.227547 20704 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:32.228106 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.190009 20565 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:32.229295 20565 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.231385 20565 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.231444 20941 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.232254 20839 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.232697 20953 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.233031 20837 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.233744 20949 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.234071 20931 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.231385 20901 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.198434 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:32.185174 20924 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.199654 18445 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 3: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:32.145907 20913 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.221082 20970 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:17:32.196244 20872 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.246461 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.264185 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:32.247781 20916 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.223326 20814 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.245464 20903 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.248821 20913 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.249711 20772 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.250268 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.159337 20963 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.244817 20891 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.199050 20914 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.267557 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.300858 18373 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:32.269146 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:32.269522 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.302065 18370 raft_consensus.cc:2376] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5. Re-sending same reply.
I20260709 12:17:32.302297 18374 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:32.280769 18441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2044) took 2457 ms (client timeout 2045 ms). Trace:
I20260709 12:17:32.281332 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.268779 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:32.303491 18441 rpcz_store.cc:269] 0709 12:17:29.806610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.817445 (+ 10835us) service_pool.cc:224] Handling call
0709 12:17:32.254295 (+2436850us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002044d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:32.263962 (+  9667us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3130220800}
I20260709 12:17:32.290550 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.291384 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.291029 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:32.293869 18448 raft_consensus.cc:2410] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:32.305145 18368 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:32.305172 18366 raft_consensus.cc:2376] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4. Re-sending same reply.
I20260709 12:17:32.304322 18369 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:32.296291 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.298133 18371 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.279153 20872 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.307741 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:32.308612 18373 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:32.308485 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.312707 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.281352 20899 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:32.314996 20936 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.318200 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.319864 18442 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:32.326807 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.327759 18370 raft_consensus.cc:2368] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:32.322721 18373 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.326195 18369 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:32.317054 20704 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.361501 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.282052 20900 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:17:32.344925 20963 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:32.346088 20966 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:17:32.349386 20920 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.340484 20755 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:32.360611 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:32.409911 18371 raft_consensus.cc:2368] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:32.334080 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.411000 18368 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:32.358621 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.413225 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.413928 18441 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:32.342483 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 1995) took 1001 ms. Trace:
I20260709 12:17:32.415632 18292 rpcz_store.cc:276] 0709 12:17:31.341518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.341623 (+   105us) service_pool.cc:224] Handling call
0709 12:17:31.363790 (+ 22167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.011761 (+647971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.342223 (+330462us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540004cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:32.342422 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464814976}
I20260709 12:17:32.335083 20898 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.342259 20979 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:32.304291 20945 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.354929 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.417742 18374 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 3: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:32.387105 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.325126 20899 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:32.368727 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.381340 20958 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:17:32.308945 20914 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.395210 18263 leader_election.cc:336] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.890s (SENT)
W20260709 12:17:32.383870 20975 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:17:32.443926 18292 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:32.444357 18292 pending_rounds.cc:85] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 5
I20260709 12:17:32.444608 18292 pending_rounds.cc:107] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:32.444880 18292 raft_consensus.cc:2889] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:32.400060 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.381348 20772 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.385334 18370 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:17:32.450911 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55732.8us
I20260709 12:17:32.451506 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.340991 20818 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.458043 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.334585 20516 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:32.424767 18373 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.441936 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.417469 20755 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.438778 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:32.445556 18448 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:32.451957 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:32.453918 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.457217 18375 raft_consensus.cc:2368] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
I20260709 12:17:32.457913 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.472302 18441 raft_consensus.cc:2410] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:32.466563 18444 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:32.436802 20899 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.468057 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.467139 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:32.467243 19008 connection.cc:256] RPC call timeout handler was delayed by -1.5120232269998723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:17:32.438522 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.479846 18299 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:32.480485 18366 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:32.469354 18373 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:32.481556 18371 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:32.481739 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.459167 18292 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:32.436880 20980 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:32.475553 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.488627 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.477118 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.449900 20936 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.474819 18296 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:32.483525 19044 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:32.491570 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.492094 18373 raft_consensus.cc:2410] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:32.492159 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 524) took 1698 ms. Trace:
I20260709 12:17:32.484853 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.493193 18373 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:32.492733 19044 rpcz_store.cc:276] 0709 12:17:30.793358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.458558 (+1665200us) service_pool.cc:224] Handling call
0709 12:17:32.483156 (+ 24598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.491847 (+  8691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.492066 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:32.494249 18373 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.482733 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.489609 18300 raft_consensus.cc:2376] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6. Re-sending same reply.
I20260709 12:17:32.495167 18440 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:32.472234 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:32.489741 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.496661 18292 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:32.489068 18291 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:32.491220 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.497208 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.445465 20945 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.497659 18298 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:32.497756 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:32.498566 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.493577 18295 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:32.499717 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 451) took 2410 ms. Trace:
I20260709 12:17:32.500337 18372 rpcz_store.cc:276] 0709 12:17:30.089154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.089424 (+   270us) service_pool.cc:224] Handling call
0709 12:17:30.100853 (+ 11429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.089663 (+1988810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.499493 (+409830us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400116554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:32.499661 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4572068224}
I20260709 12:17:32.498689 20697 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:32.492800 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.505980 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.506629 18366 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.504053 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.507609 18292 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.507795 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.500877 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.483417 20846 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.519726 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.523689 19039 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:32.520148 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.524940 18300 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:32.522251 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.526485 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:32.529275 18441 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:32.488488 20980 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:32.530503 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.531024 18371 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:32.476640 20941 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.532281 18295 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:32.529238 19037 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.465637 20772 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.538329 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.539716 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:32.505468 20837 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.561616 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.558585 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.521102 20697 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:32.564956 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.568444 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.569048 18299 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.569839 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.566498 18366 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:32.562644 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:32.568768 19041 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.572223 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.572651 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.572805 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:32.573182 19041 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:32.574231 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.575587 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:32.575693 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.574950 19041 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:32.577757 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:32.579288 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:32.526503 20846 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.583539 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.534178 20973 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.587435 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.578601 18371 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:32.590807 20998 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.593636 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.597050 18447 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:32.534946 20941 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.611398 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.535223 20900 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.612771 18300 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.621218 19040 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:32.623435 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:32.519799 20945 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.504324 20936 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.628185 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:32.630131 18291 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:32.631521 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.628947 18445 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:32.530717 20980 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.642113 18291 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:32.595456 20846 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.543313 20924 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.669914 18299 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.675374 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:32.676136 18291 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:32.685214 18299 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:32.689323 20994 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.109s	user 0.001s	sys 0.001s
I20260709 12:17:32.624686 20998 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.535166 20944 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:32.691906 18445 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:32.710366 18292 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:32.734005 18300 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:32.736148 18292 pending_rounds.cc:85] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 4
I20260709 12:17:32.736486 18292 pending_rounds.cc:107] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:32.736897 18292 raft_consensus.cc:2889] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:32.714485 19037 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:32.659848 20974 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 12:17:32.749977 19041 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:32.574505 20837 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.689520 20704 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.668164 20941 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:32.624171 20973 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:32.778213 18325 heartbeater.cc:646] Failed to heartbeat to 127.16.193.126:46027 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.16.193.126:46027 timed out after 15.000s (SENT)
W20260709 12:17:32.781476 20984 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 12:17:32.764400 20980 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:32.785881 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:32.625335 20900 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.784457 20936 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.762106 20898 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:32.868008 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82269.5us
I20260709 12:17:32.885185 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:32.886374 18338 leader_election.cc:400] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:17:32.886746 18338 leader_election.cc:403] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:32.888664 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:32.889748 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:32.887216 19038 raft_consensus.cc:2376] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4. Re-sending same reply.
I20260709 12:17:32.898882 20900 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.899592 20900 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.900015 20900 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.900415 20900 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.792640 20941 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.904425 20846 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:32.905230 20941 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:32.905821 20941 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:32.906194 20941 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:32.906587 20941 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:32.907706 20898 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:32.908849 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:32.909168 20837 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:32.909487 18445 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.911408 20973 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:32.913713 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.919732 20891 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:32.820950 20818 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:32.923396 20846 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:32.843945 20814 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.003s
I20260709 12:17:32.821543 20939 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:32.885370 20994 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:17:32.926246 20673 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.032542 18296 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.000s
I20260709 12:17:33.035624 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.044956 18296 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:17:33.045110 18447 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:33.063759 18445 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:33.017829 20772 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.035874 20673 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:33.085294 20673 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.089465 20944 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:33.090723 20945 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:33.096889 20872 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.779s	user 0.009s	sys 0.000s
I20260709 12:17:33.097481 20872 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.099002 18367 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:33.100936 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:33.101756 18297 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.000s
I20260709 12:17:33.102635 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:33.106389 20755 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.600s	user 0.005s	sys 0.004s
I20260709 12:17:33.106938 20755 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:33.108376 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.114943 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.115144 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.115993 18372 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:33.116977 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.118276 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.115985 18370 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:33.118932 18372 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:33.121760 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:33.121881 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:33.122599 18371 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.126045 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.128826 18375 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:33.129853 18374 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:33.131922 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.132661 18370 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:33.133618 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.120402 20322 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.112928 20649 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:33.137485 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 246386us
I20260709 12:17:33.147683 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.148488 20772 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:33.148723 18370 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:33.149024 20772 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.149581 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.151080 20772 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:33.122911 20755 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.151886 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.153964 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.154615 18375 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:33.155440 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:33.156750 18298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.015s	sys 0.016s
W20260709 12:17:33.158905 18298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.016s	sys 0.017s
I20260709 12:17:33.159025 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.159793 21001 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:33.195015 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.161007 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.197005 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:33.163443 20959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.004s	sys 0.028s
W20260709 12:17:33.207813 20899 consensus_meta.cc:287] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.002s
W20260709 12:17:33.164871 20657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.023s	sys 0.026s
I20260709 12:17:33.213214 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.215917 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.217252 18368 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:33.201917 20959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.004s	sys 0.028s
I20260709 12:17:33.151834 20322 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.152923 20649 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:33.224589 18369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.008s	sys 0.045s
I20260709 12:17:33.194466 20872 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.229620 18371 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:33.210071 21001 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:33.230659 18369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.009s	sys 0.045s
W20260709 12:17:33.211264 20992 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:33.231030 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.236819 18375 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:33.214100 20657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.023s	sys 0.027s
W20260709 12:17:33.233760 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 155011us
I20260709 12:17:33.234645 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.251775 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.252063 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.252468 19044 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:33.252833 19039 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.255267 18368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35973fcee55042b284c6f62e2681c304"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:33.256086 18368 raft_consensus.cc:606] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:33.256518 18368 raft_consensus.cc:614] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:33.265281 18375 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:33.212595 20899 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.236207 21001 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.254419 20872 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.271852 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.241459 20704 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:33.274008 18373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.018s	sys 0.050s
I20260709 12:17:33.274072 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.277148 21001 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:33.287896 18371 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:33.288342 18373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.018s	sys 0.050s
I20260709 12:17:33.291272 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.291924 19044 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:33.292814 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.294184 19044 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:33.294849 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.296422 20649 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.276667 21011 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.286358 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.286412 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.307830 20657 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:33.310029 20657 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.310431 20657 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:33.311160 20322 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.314535 20891 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
W20260709 12:17:33.308720 20697 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.005s
I20260709 12:17:33.315126 20902 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.315349 20697 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:33.317389 20697 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.284883 20959 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.319482 20959 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:17:33.320501 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.321485 20959 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:33.318710 20673 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:33.322110 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.324425 18366 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:33.324993 19039 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:33.325472 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.331049 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:33.352501 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0906101690000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:17:33.363963 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0616066919999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.36440000000000006s for a response
I20260709 12:17:33.338187 18415 leader_election.cc:400] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:33.365908 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:33.336194 18372 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:33.368945 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:33.371394 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.388387 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:33.390308 21006 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:17:33.388474 20772 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.395396 21014 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:17:33.395406 20772 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.395968 20772 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.370611 20649 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:33.383117 20944 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.404453 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.406688 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:33.391083 21000 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:33.433288 21001 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.435638 21001 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.436365 20649 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:33.436785 20649 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.437089 20649 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:33.407450 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.443104 21001 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:33.443620 21001 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:33.443446 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.443935 20673 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.444357 19039 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:33.390519 20926 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:33.445088 20926 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.444617 21001 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.391199 20917 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:17:33.407950 20944 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:33.489097 19039 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:33.490911 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.483512 20944 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:33.528931 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 160065us
I20260709 12:17:33.531620 18415 leader_election.cc:400] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:33.532011 18415 leader_election.cc:403] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:33.533018 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.485280 20872 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:33.493144 20673 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:33.542484 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 154123us
I20260709 12:17:33.544050 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.545020 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.537662 20899 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.547526 20673 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:33.558897 20673 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.550884 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:33.575543 21023 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.576299 21023 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.576635 21023 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.576932 21023 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:33.577268 20900 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.676s	user 0.008s	sys 0.003s
W20260709 12:17:33.582038 20941 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
W20260709 12:17:33.559202 21008 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:17:33.582424 20941 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:33.560038 21013 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:17:33.560360 21004 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 12:17:33.603588 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86613.3us
I20260709 12:17:33.548966 21022 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.559646 21016 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:17:33.570900 20926 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
W20260709 12:17:33.618075 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73046.5us
I20260709 12:17:33.622869 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:33.603477 20917 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.626144 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.627594 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.628738 18369 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.629231 18369 raft_consensus.cc:614] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:33.604676 20516 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:33.607553 20772 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.577706 20900 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.570879 20657 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.622175 21022 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.560105 20649 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.638644 18375 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:33.641376 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1559) took 1232 ms. Trace:
W20260709 12:17:33.633220 21024 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:17:33.649176 18370 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:33.643337 18373 rpcz_store.cc:276] 0709 12:17:32.408483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.494931 (+ 86448us) service_pool.cc:224] Handling call
0709 12:17:32.495216 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:32.495560 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.495585 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:32.519503 (+ 23918us) log.cc:844] Serialized 52 byte log entry
0709 12:17:33.289802 (+770299us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:33.289856 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.289961 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.641113 (+351152us) raft_consensus.cc:1685] finished
0709 12:17:33.641139 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.641312 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":1599,"raft.run_wall_time_us":21055,"thread_start_us":703911,"threads_started":1,"wal-append.queue_time_us":847809}
I20260709 12:17:33.650816 18373 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 12:17:33.637315 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 104394us
I20260709 12:17:33.653540 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.657253 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:33.644381 21022 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.627853 20872 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.649958 20926 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.675239 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 47708.6us
I20260709 12:17:33.649807 21027 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:33.632053 20917 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.685369 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 545) took 1327 ms. Trace:
I20260709 12:17:33.685734 18442 rpcz_store.cc:276] 0709 12:17:32.358417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.368888 (+ 10471us) service_pool.cc:224] Handling call
0709 12:17:32.369330 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.383275 (+ 13945us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:32.404665 (+ 21390us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.750772 (+346107us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b1800436720. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:17:33.231044 (+480272us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.242248 (+ 11204us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.242390 (+   142us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:33.242427 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.242496 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.685113 (+442617us) raft_consensus.cc:1685] finished
0709 12:17:33.685193 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.685317 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19193,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":1998,"raft.queue_time_us":4004,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":18105,"spinlock_wait_cycles":17944448,"thread_start_us":480022,"threads_started":1,"wal-append.queue_time_us":931741}
I20260709 12:17:33.684584 20649 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.672067 21018 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.635535 20516 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.678454 20926 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:33.692123 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.710059 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.691740 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.711977 18442 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:33.694471 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.694792 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.695048 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.695200 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:33.713011 19039 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.695583 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.695984 20657 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.716794 21023 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:33.697273 18339 leader_election.cc:336] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.554s (SENT)
I20260709 12:17:33.698544 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.684602 20900 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.717932 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:33.704188 21026 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:33.720314 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.721311 18444 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:33.704095 21018 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.704945 20941 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.679816 21027 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.692061 20772 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.706300 20649 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:33.726961 20917 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.727702 20649 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:33.727962 21018 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:33.728777 20649 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:33.729902 20924 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:33.730181 20649 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.730537 20924 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.731410 21027 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.722373 21026 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:33.732920 21026 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.733850 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.734508 18441 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:33.691476 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:33.711045 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.735376 18447 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.713791 18440 raft_consensus.cc:2376] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3. Re-sending same reply.
I20260709 12:17:33.716070 20944 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.716565 21022 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:33.696110 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.718134 20926 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.726328 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:33.742717 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53640.5us
I20260709 12:17:33.750655 19039 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
W20260709 12:17:33.737707 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46290.5us
I20260709 12:17:33.725284 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.730688 20649 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:33.751858 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:33.753309 18441 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.753227 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.753902 18442 raft_consensus.cc:2368] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4.
I20260709 12:17:33.755600 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.756891 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.757193 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.757483 18441 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:33.766063 18447 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:33.783700 18448 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:33.848310 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 96561.3us
I20260709 12:17:33.945833 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.968362 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:33.968782 18366 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:33.864792 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:33.922083 18367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1532) took 1655 ms (client timeout 1819 ms). Trace:
W20260709 12:17:33.970054 18367 rpcz_store.cc:269] 0709 12:17:32.266770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.266884 (+   114us) service_pool.cc:224] Handling call
0709 12:17:33.921840 (+1654956us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54003f1454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:33.922000 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825781504}
I20260709 12:17:33.898072 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:33.773746 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:33.757701 18442 raft_consensus.cc:2376] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4. Re-sending same reply.
I20260709 12:17:33.758463 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.758785 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:33.983644 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.984913 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:33.984853 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:33.984504 18440 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:34.009358 18442 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:34.165457 18264 connection.cc:256] RPC call timeout handler was delayed by -1.2833684349998293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 12:17:34.174135 18264 connection.cc:256] RPC call timeout handler was delayed by -1.1222684349997962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:17:34.035063 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:33.989602 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.175800 18366 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:34.174976 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.178901 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:34.093391 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 532) took 1537 ms. Trace:
I20260709 12:17:34.096855 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.177209 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.095553 18368 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.986896 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:33.986932 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 213281us
I20260709 12:17:34.184715 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:33.843793 20980 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.968s	user 0.010s	sys 0.003s
W20260709 12:17:34.033928 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 135971us
I20260709 12:17:33.864516 20845 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.180672 19041 rpcz_store.cc:276] 0709 12:17:32.555676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.577520 (+ 21844us) service_pool.cc:224] Handling call
0709 12:17:32.749650 (+172130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.750594 (+   944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.093190 (+1342596us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400611050. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.093323 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370361088}
W20260709 12:17:34.186375 19038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 761) took 1647 ms (client timeout 1993 ms). Trace:
I20260709 12:17:34.188405 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1627) took 2797 ms. Trace:
I20260709 12:17:33.876470 20998 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:33.781220 21023 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:33.856936 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 133704us
I20260709 12:17:33.779922 21022 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.182904 18373 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:33.920151 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 690) took 2417 ms. Trace:
I20260709 12:17:33.858850 21033 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:33.958710 21039 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:33.775202 20516 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:34.212953 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.268383 18372 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:34.215996 18374 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:34.216172 18449 rpcz_store.cc:276] 0709 12:17:31.349705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.349831 (+   126us) service_pool.cc:224] Handling call
0709 12:17:31.350188 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.642576 (+292388us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:32.022345 (+379769us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.082218 (+ 59873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.082289 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:32.082361 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.068129 (+985768us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b18003ff700. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419
0709 12:17:34.145841 (+1077712us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400355cd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:34.146188 (+   347us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.146964 (+   776us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":484625,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":10649,"raft.queue_time_us":95,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5734,"spinlock_wait_cycles":692272768,"thread_start_us":411282,"threads_started":2,"wal-append.queue_time_us":307969,"wal-append.run_cpu_time_us":2479,"wal-append.run_wall_time_us":1430919}
I20260709 12:17:34.226286 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.270599 18371 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:34.232226 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:34.271418 18442 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:34.233110 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:34.215132 19038 rpcz_store.cc:269] 0709 12:17:32.538943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.539079 (+   136us) service_pool.cc:224] Handling call
0709 12:17:34.186200 (+1647121us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54006bd854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.186306 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727019392}
I20260709 12:17:34.090059 20967 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:34.224053 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 237346us
I20260709 12:17:33.986342 20973 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.274s
W20260709 12:17:34.262393 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 294041us
I20260709 12:17:34.251832 20845 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:33.986212 20943 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.113408 21040 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.132947 20944 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.034735 20509 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.189510 20847 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:34.274029 20967 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.780395 21036 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.472s
I20260709 12:17:34.272166 19039 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:33.892858 20926 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.212638 21044 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:34.247044 18366 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:34.032707 20772 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.277483 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:34.325831 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 88616.7us
W20260709 12:17:34.278837 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 100026us
I20260709 12:17:34.266608 21033 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:34.279716 18372 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:34.280242 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 88996.5us
I20260709 12:17:34.316867 20845 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:34.322451 20967 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:34.314079 21037 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
I20260709 12:17:34.233687 21041 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:17:34.314855 19042 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 3.
I20260709 12:17:34.213701 21034 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:34.008203 20914 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.275055 20649 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:34.318971 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 621661us
W20260709 12:17:34.323168 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 336325us
I20260709 12:17:34.371821 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:34.318110 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 467) took 1043 ms. Trace:
I20260709 12:17:34.315419 21040 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:34.328662 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 573120us
W20260709 12:17:34.329511 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52134.3us
I20260709 12:17:34.278604 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 529) took 1724 ms. Trace:
I20260709 12:17:34.329794 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.034868 20946 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.261s
I20260709 12:17:34.316469 20847 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.226365 21038 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 12:17:34.339704 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:34.403267 19037 raft_consensus.cc:2376] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5. Re-sending same reply.
I20260709 12:17:34.404356 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:34.404963 19037 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:34.340883 18447 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:34.342362 19044 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:17:34.344712 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 694) took 1855 ms (client timeout 2014 ms). Trace:
I20260709 12:17:34.347175 20516 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 12:17:34.406340 19042 rpcz_store.cc:269] 0709 12:17:32.489236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.489343 (+   107us) service_pool.cc:224] Handling call
0709 12:17:34.184560 (+1695217us) spinlock_profiling.cc:243] Waited 962 us on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:34.344659 (+160099us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213632}
I20260709 12:17:34.331878 20772 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.407081 21053 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:34.407547 20772 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.408424 21053 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:34.409659 21053 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:34.336002 20704 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:34.410319 20704 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:34.411108 20704 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.347904 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:34.349565 20926 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.344935 20973 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.412067 20967 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.412839 20973 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.413512 20845 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.413940 20845 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:34.414212 20967 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.414299 20845 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.414814 20973 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.350411 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:34.404387 20847 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:34.350812 21036 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:34.344602 21047 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:34.258721 19036 rpcz_store.cc:276] 0709 12:17:31.503214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.534739 (+ 31525us) service_pool.cc:224] Handling call
0709 12:17:31.987693 (+452954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.084867 (+ 97174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.084929 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.085634 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.186033 (+100399us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.188617 (+  2584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.196147 (+  7530us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.411576 (+215429us) raft_consensus.cc:1685] finished
0709 12:17:32.413447 (+  1871us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.919808 (+1506361us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400545bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:33.920070 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14300,"raft.queue_time_us":84626,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4002,"spinlock_wait_cycles":727389440,"thread_start_us":158976,"threads_started":2,"wal-append.queue_time_us":302873,"wal-append.run_cpu_time_us":1686,"wal-append.run_wall_time_us":1032038}
I20260709 12:17:34.336915 20982 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:34.360368 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1676) took 1243 ms. Trace:
I20260709 12:17:34.423471 18374 rpcz_store.cc:276] 0709 12:17:33.116790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.136692 (+ 19902us) service_pool.cc:224] Handling call
0709 12:17:34.360183 (+1223491us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400395154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.360310 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425028480}
W20260709 12:17:34.361971 21026 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.629s	user 0.001s	sys 0.008s
I20260709 12:17:34.363386 20998 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:34.367730 18299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2004) took 1805 ms (client timeout 1884 ms). Trace:
W20260709 12:17:34.428514 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 153593us
W20260709 12:17:34.428889 18299 rpcz_store.cc:269] 0709 12:17:32.561826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.569631 (+  7805us) service_pool.cc:224] Handling call
0709 12:17:34.367565 (+1797934us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b54001cd9d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.367671 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208590592}
I20260709 12:17:34.372301 18375 rpcz_store.cc:276] 0709 12:17:33.275424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.656646 (+381222us) service_pool.cc:224] Handling call
0709 12:17:33.660268 (+  3622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.660508 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:33.668326 (+  7818us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.707330 (+ 39004us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.317530 (+610200us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18002c8f60. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:17:34.317691 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.317751 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:34.317805 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.317904 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.318040 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":613432,"prepare.queue_time_us":9533,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":1572,"raft.queue_time_us":616015,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":10570,"spinlock_wait_cycles":50692480,"thread_start_us":38367,"threads_started":1,"wal-append.queue_time_us":128583}
W20260709 12:17:34.403777 21027 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.672s	user 0.008s	sys 0.004s
I20260709 12:17:34.326877 20900 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.389429 19040 rpcz_store.cc:276] 0709 12:17:32.554456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.576858 (+ 22402us) service_pool.cc:224] Handling call
0709 12:17:32.577216 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.621788 (+ 44572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.278338 (+1656550us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b5400748bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:34.278535 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44834048}
I20260709 12:17:34.381747 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1600) took 1858 ms. Trace:
I20260709 12:17:34.392688 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:34.398859 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.405875 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:34.406363 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76642.4us
I20260709 12:17:34.316207 20943 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.441798 20998 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.442248 20943 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.443028 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 462) took 1305 ms. Trace:
I20260709 12:17:34.356736 21033 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.443765 21033 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.443846 20998 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.371121 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.400547 21040 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.401171 20899 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.437588 18292 rpcz_store.cc:276] 0709 12:17:32.523320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.525333 (+  2013us) service_pool.cc:224] Handling call
0709 12:17:32.525652 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.750363 (+224711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.381470 (+1631107us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54000717d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:34.381676 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423674624}
I20260709 12:17:34.347127 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:34.472644 19038 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:34.419317 21053 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:34.444656 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 73659.7us
W20260709 12:17:34.373798 18339 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.675s (SENT)
I20260709 12:17:34.471469 20998 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:34.473827 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 100068us
I20260709 12:17:34.404121 21018 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:34.449646 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43831.7us
I20260709 12:17:34.443430 18370 rpcz_store.cc:276] 0709 12:17:33.137289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.149337 (+ 12048us) service_pool.cc:224] Handling call
0709 12:17:34.442859 (+1293522us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540009f6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.442975 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147032448}
W20260709 12:17:34.450604 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51833.6us
I20260709 12:17:34.465001 21048 raft_consensus.cc:2764] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:34.512183 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:34.512467 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 82799us
I20260709 12:17:34.506976 19043 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:34.481103 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:34.448741 21027 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.513335 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:34.471439 20926 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:34.512948 18339 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.701s (SENT)
I20260709 12:17:34.401985 21034 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:34.515947 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 123362us
I20260709 12:17:34.526283 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.516029 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:34.521054 18441 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:34.525475 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:34.568645 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:34.516916 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:34.465874 21000 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.009s	sys 0.002s
I20260709 12:17:34.543824 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:34.521672 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:34.572221 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:34.573565 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 57607.8us
W20260709 12:17:34.573108 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46893.2us
I20260709 12:17:34.575735 20901 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.578889 20901 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.579310 20901 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.440209 21026 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.435216 20509 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:34.577423 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:34.581278 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:34.580806 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:34.627442 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46746.9us
W20260709 12:17:34.628978 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 112094us
I20260709 12:17:34.630277 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:34.631767 20980 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:34.632481 20891 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:34.633430 20980 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:34.634091 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52879.8us
I20260709 12:17:34.634219 21041 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.634871 21041 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.634550 20980 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.635269 21041 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.629467 20649 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:34.637115 20649 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.637586 20649 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:34.630342 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 117412us
I20260709 12:17:34.420125 21055 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.169s
I20260709 12:17:34.640893 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:34.527895 21052 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:34.588510 21000 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.085s	user 0.009s	sys 0.005s
I20260709 12:17:34.571483 20917 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:34.521508 20926 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:34.516546 20924 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.000s
I20260709 12:17:34.660924 18443 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:34.474118 20998 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:34.588397 21039 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.633447 20891 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:34.661418 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 326836us
I20260709 12:17:34.588742 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:34.679543 20845 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.680454 20818 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:34.641901 20509 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.661545 20943 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:34.659782 21000 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:34.684485 18370 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:34.435985 20982 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.626048 20516 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.640316 21047 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.626333 21048 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.686168 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.715822 18373 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:34.716346 18373 raft_consensus.cc:606] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:34.716638 18373 raft_consensus.cc:614] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:34.687556 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98885us
I20260709 12:17:34.718706 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:34.688223 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 110875us
I20260709 12:17:34.721555 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:34.571316 20914 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:34.689039 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.723451 18366 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:34.567217 21018 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.662251 20926 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.705651 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.706310 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.687351 20917 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:34.684136 21052 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:34.733067 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 104062us
I20260709 12:17:34.690068 20898 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:34.689891 20900 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:34.704118 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.738894 18370 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:34.739811 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.709267 19041 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:34.687285 21053 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.515893 21054 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.588176 21036 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:17:34.688686 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:34.713349 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 197899us
I20260709 12:17:34.715309 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.727864 18368 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:34.729396 18371 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:34.663883 21039 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.516173 20899 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.734329 18369 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:34.734969 18375 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:34.736515 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.754925 18366 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:34.688483 20891 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.719592 21048 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.660208 20772 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.713102 20818 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:34.719347 21047 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:34.750682 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:34.694330 20924 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:34.714454 20516 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.726763 20972 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:34.757627 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 232219us
I20260709 12:17:34.734804 20946 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.746268 20917 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.774928 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.775900 18371 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:34.776270 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:34.774370 18369 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:17:34.782414 18366 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:34.753484 21039 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.780743 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:34.761135 20891 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.798321 18446 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:34.757778 21067 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.767411 21048 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.747773 21054 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:34.808465 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.772632 21047 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.753685 20914 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:17:34.777184 21052 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.768492 21061 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:17:34.747772 20900 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.867303 20900 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.747551 20898 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:34.868832 21060 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 12:17:34.869735 20898 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.773658 20516 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.875388 18372 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:34.878497 21027 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.767606 21055 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:17:34.879963 21055 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:34.880402 21055 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:34.881372 21055 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.882921 21055 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.797273 20980 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:34.794507 21065 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.846112 20772 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.794816 20998 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:34.847733 20845 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.847275 20818 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.849428 21052 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.773892 20972 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.860119 21050 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:34.769164 21066 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 12:17:34.884833 21059 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:17:34.886412 21018 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:34.890851 20509 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.896943 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2062) took 1203 ms. Trace:
I20260709 12:17:34.994113 18443 rpcz_store.cc:276] 0709 12:17:33.693871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.693990 (+   119us) service_pool.cc:224] Handling call
0709 12:17:34.896592 (+1202602us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.896862 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118233472}
I20260709 12:17:34.899152 20956 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:34.908315 20973 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:34.831712 20944 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.830729 21054 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.932686 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 569) took 1185 ms. Trace:
I20260709 12:17:35.017136 18441 rpcz_store.cc:276] 0709 12:17:33.746926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.758152 (+ 11226us) service_pool.cc:224] Handling call
0709 12:17:34.932501 (+1174349us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54002585d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.932630 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234958208}
W20260709 12:17:34.932639 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 151966us
W20260709 12:17:34.933259 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 203976us
W20260709 12:17:34.917616 21044 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.580s	user 0.003s	sys 0.006s
I20260709 12:17:34.938575 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 570) took 1191 ms. Trace:
I20260709 12:17:35.025817 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:34.941445 18372 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:34.835788 21064 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:34.960058 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 571) took 1211 ms. Trace:
W20260709 12:17:34.949364 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 170331us
I20260709 12:17:35.035511 18447 rpcz_store.cc:276] 0709 12:17:33.748334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.983372 (+235038us) service_pool.cc:224] Handling call
0709 12:17:34.959880 (+976508us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400423e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.959987 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818582784}
I20260709 12:17:35.036854 18292 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:35.037576 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.037748 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.038023 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.038133 18294 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:35.003818 21034 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:34.932132 20941 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:34.945278 20898 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.039497 20941 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.039867 20898 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.041700 20898 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.042591 18298 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:34.945096 21026 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.043143 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.043581 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.039441 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.044417 18291 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:35.044728 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.045239 18295 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.007771 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:35.048408 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:35.048760 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.049118 18366 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.050423 20837 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:35.050752 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:35.052309 20837 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.055984 20982 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.025688 20924 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.962622 20972 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.057252 20924 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.026654 18367 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:35.031531 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:35.032095 21044 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:35.032478 21036 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:35.060774 21036 consensus_queue.cc:1243] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:17:35.061208 21036 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:35.061575 21036 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:35.062762 21052 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:35.036800 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.063573 18296 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.038254 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:35.006387 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.064713 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:35.066068 18447 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:35.066766 18367 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:35.041713 20941 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:34.963577 21050 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:35.069067 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:35.068557 21050 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.069676 18444 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:35.042491 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.070281 21050 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.043741 18292 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.071592 18447 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:35.062341 21036 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:34.936412 20980 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.026034 18446 rpcz_store.cc:276] 0709 12:17:33.747637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.854276 (+106639us) service_pool.cc:224] Handling call
0709 12:17:34.929803 (+1075527us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54003b1fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:34.938505 (+  8702us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905099136}
I20260709 12:17:35.023111 18369 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:35.020552 18375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:35.084700 18375 raft_consensus.cc:606] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.085029 18375 raft_consensus.cc:614] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:35.075315 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:35.075992 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.086566 18299 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:35.077572 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:34.999323 20900 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.998062 20956 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:35.061367 20937 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:35.016414 20944 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.012555 20973 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:35.108476 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:35.108492 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.109305 18440 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:35.109200 18299 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:35.110116 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:35.110731 19041 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 3.
I20260709 12:17:34.801643 21000 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.116057 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:35.116416 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:35.108168 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:35.116909 18448 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.120101 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:35.120702 18440 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
W20260709 12:17:35.123706 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:35.124387 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.125092 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:35.127472 18295 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:35.122850 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:35.123780 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:35.129591 18442 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:35.145756 18366 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:35.131569 18292 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:35.148717 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.089486 21034 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:35.153731 18339 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.014s	sys 0.012s
I20260709 12:17:35.108682 20944 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.156076 18339 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.014s	sys 0.012s
W20260709 12:17:35.157832 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 518025us
I20260709 12:17:35.160367 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.161114 18292 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.161849 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:35.166877 18369 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:35.166977 18446 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:35.112130 21036 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:35.168913 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:35.157771 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:35.170610 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:35.174723 18366 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:35.176172 18339 leader_election.cc:336] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.825s (SENT)
W20260709 12:17:35.179430 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 446952us
W20260709 12:17:35.179896 18339 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.058s (SENT)
I20260709 12:17:35.184829 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.194470 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.196131 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:35.199630 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:35.201701 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.203919 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.206069 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.207715 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:35.208781 19035 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:35.175333 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:35.220032 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:35.221227 18197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 210) took 17733 ms (client timeout 15000 ms). Trace:
I20260709 12:17:35.221972 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.222779 18292 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:35.225697 18375 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:17:35.156762 20995 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:35.163856 21034 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:35.270467 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:35.262457 18448 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
W20260709 12:17:35.264627 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 189384us
I20260709 12:17:35.114061 20900 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:35.266134 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 86250.9us
I20260709 12:17:35.121413 21049 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:17:35.170006 20982 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:35.171730 21018 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:35.271899 18339 leader_election.cc:336] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.065s (SENT)
W20260709 12:17:35.288048 18339 leader_election.cc:336] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.352s (SENT)
I20260709 12:17:35.288579 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:35.290527 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 128905us
I20260709 12:17:35.152558 21026 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 12:17:35.272701 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:35.226903 21058 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.279418 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.137017 21065 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 12:17:35.292210 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:35.271324 18367 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:35.278563 21034 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.261269 20967 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:35.176512 20943 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 12:17:35.261026 21027 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:35.145404 21039 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:35.221789 18197 rpcz_store.cc:269] 0709 12:17:17.446190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.446422 (+   232us) service_pool.cc:224] Handling call
0709 12:17:35.179556 (+17733134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:31.163665 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:31.164517 (+   852us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:31.164582 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:17:31.164605 (+    23us) tablet.cc:662] Decoding operations
0709 12:17:31.253283 (+ 88678us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:31.253364 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:31.253392 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:17:31.254031 (+   639us) tablet.cc:701] Row locks acquired
0709 12:17:31.254075 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:17:31.254231 (+   156us) write_op.cc:270] Start()
0709 12:17:31.254372 (+   141us) write_op.cc:276] Timestamp: P: 1783599451254173 usec, L: 0
0709 12:17:31.254388 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:17:31.444425 (+190037us) log.cc:844] Serialized 2448 byte log entry
0709 12:17:32.339858 (+895433us) op_driver.cc:464] REPLICATION: finished
0709 12:17:32.531686 (+191828us) write_op.cc:301] APPLY: starting
0709 12:17:32.531804 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:32.532048 (+   244us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:32.532204 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:32.611064 (+ 78860us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:32.632693 (+ 21629us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:17:32.632744 (+    51us) write_op.cc:312] APPLY: finished
0709 12:17:32.689401 (+ 56657us) log.cc:844] Serialized 53 byte log entry
0709 12:17:32.696416 (+  7015us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.696540 (+   124us) write_op.cc:454] Released schema lock
0709 12:17:32.696585 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:32.696992 (+   407us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.697021 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9214,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":191512,"apply.run_cpu_time_us":3447,"apply.run_wall_time_us":165314,"num_ops":6,"prepare.queue_time_us":711671,"prepare.run_cpu_time_us":12410,"prepare.run_wall_time_us":561928,"raft.queue_time_us":209342,"raft.run_cpu_time_us":12523,"raft.run_wall_time_us":140836,"replication_time_us":1085251,"thread_start_us":428781,"threads_started":4,"wal-append.queue_time_us":571034,"wal-append.run_cpu_time_us":23777,"wal-append.run_wall_time_us":1755902}]]}
I20260709 12:17:35.264329 20995 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:35.320946 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.044s
W20260709 12:17:35.351394 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.005s	sys 0.045s
W20260709 12:17:35.351926 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 663301us
I20260709 12:17:35.277455 18366 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:35.326752 18445 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:35.326856 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.326989 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.353894 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:35.354575 18448 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:35.298482 21075 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:35.334362 20898 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:17:35.307600 19041 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:35.324311 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:35.268934 20924 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.261462 20772 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:17:35.296294 21018 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:35.348181 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75536.6us
I20260709 12:17:35.299759 21053 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:35.333504 20941 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:17:35.325675 21058 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.353581 20995 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.360008 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:35.335925 21042 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:35.322187 21076 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:35.363272 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:35.352401 21027 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.364398 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:35.402060 19044 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 12:17:35.402765 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.403576 19037 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:35.365070 21018 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.331743 20982 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:35.331403 21026 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:35.319769 20814 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:17:35.265683 20956 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.359131 20704 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:35.420917 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.369694 21072 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.392576 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:35.360558 20967 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:35.321684 21073 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:17:35.371917 21053 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:35.413261 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50089.3us
I20260709 12:17:35.438593 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:35.418781 20901 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.832s	user 0.007s	sys 0.003s
I20260709 12:17:35.399601 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:35.384399 20847 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.968s	user 0.010s	sys 0.000s
W20260709 12:17:35.383735 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59489.7us
I20260709 12:17:35.424113 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:35.394838 20924 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:35.399926 20516 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.005s
I20260709 12:17:35.413956 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.411983 21026 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:35.440984 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.412350 20982 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:35.452952 18291 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.395114 18196 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:35.482815 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.423075 21072 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:35.396704 21027 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.438139 20956 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.484331 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:35.452092 20891 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
I20260709 12:17:35.485312 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.428385 20967 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:35.483752 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.435561 21076 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:35.492167 18295 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
I20260709 12:17:35.484877 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.443434 21053 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:35.393973 21087 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:17:35.453353 20924 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:35.435470 21095 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:35.451570 21071 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:35.529443 21085 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:17:35.529794 21071 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:35.492632 21072 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.530875 21071 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.524698 18293 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.534821 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.535356 18293 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.535475 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:35.536549 20649 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.898s	user 0.007s	sys 0.003s
I20260709 12:17:35.536723 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:35.451478 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:35.443748 21068 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 12:17:35.541996 21033 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.098s	user 0.002s	sys 0.006s
I20260709 12:17:35.529045 20924 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:35.486862 20837 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:35.537007 20649 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:35.576156 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.576951 18294 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:35.525833 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.543488 21026 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.454320 20847 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.547369 20967 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.485965 20982 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.561565 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.609221 18295 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.583187 18291 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:35.544711 18293 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:35.484499 20516 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.447789 20901 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.524266 20891 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.490095 21075 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
W20260709 12:17:35.565217 21047 consensus_meta.cc:287] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.726s	user 0.011s	sys 0.000s
W20260709 12:17:35.660759 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 1a92f90eb0424e3d870764a2a328e508: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:35.663658 18292 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.676077 19042 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:35.685004 18299 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:35.606107 20900 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.559163 20972 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.689941 18374 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:35.700465 20891 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:35.701351 21048 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.867s	user 0.006s	sys 0.002s
I20260709 12:17:35.701776 21048 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.703459 21072 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.704394 20901 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.705510 21072 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.543505 21095 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:35.720161 21095 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.668349 20980 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.628355 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:35.724710 18294 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.628753 18440 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:35.595901 18447 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:35.575817 20704 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:35.645762 21033 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 12:17:35.658778 20937 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.000s
I20260709 12:17:35.706914 19044 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
W20260709 12:17:35.667972 21082 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
W20260709 12:17:35.730372 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 90144.2us
I20260709 12:17:35.661324 20967 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:35.734895 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.002s	sys 0.004s
W20260709 12:17:35.783776 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.002s	sys 0.004s
W20260709 12:17:35.735394 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 366619us
W20260709 12:17:35.739615 20926 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.992s	user 0.010s	sys 0.000s
I20260709 12:17:35.744627 20972 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.745216 19036 raft_consensus.cc:2368] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4.
I20260709 12:17:35.689201 21075 consensus_queue.cc:1243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:17:35.746969 21050 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.005s
I20260709 12:17:35.747460 21058 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:35.748582 21040 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.233s	user 0.006s	sys 0.003s
W20260709 12:17:35.749243 21055 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.000s
I20260709 12:17:35.749909 20899 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:35.752112 21036 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.000s
I20260709 12:17:35.706763 21039 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:35.667531 21047 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:35.753355 21041 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.118s	user 0.007s	sys 0.003s
I20260709 12:17:35.738602 20516 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.755218 21048 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.759786 20995 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:35.784083 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 611896us
I20260709 12:17:35.869552 20926 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.761268 20847 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:35.870754 18449 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:35.763574 20657 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:35.871379 20657 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:35.871129 21041 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.871917 20926 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:35.872205 20657 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:35.855789 20944 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.000s
I20260709 12:17:35.873271 20657 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.873708 20944 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.875777 20944 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.888769 21040 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.891371 18444 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:35.761055 21044 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:35.895773 21044 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:35.849963 20924 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.896857 21044 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.898176 21044 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:35.858794 21018 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.905599 21018 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:35.789359 21054 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.000s
I20260709 12:17:35.867439 21047 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:35.933090 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.859925 21027 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.870061 19040 raft_consensus.cc:2368] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:35.737739 20649 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.762456 21034 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.848448 21053 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:35.898967 20924 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:35.977675 21027 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:35.900493 21040 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.984858 21034 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.976421 21054 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.854794 20982 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:35.992022 21053 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:35.993837 20982 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.001376 21054 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:36.053580 21000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.011s	sys 0.044s
W20260709 12:17:36.056098 21000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.011s	sys 0.044s
I20260709 12:17:36.058446 19043 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:35.691097 18196 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 2, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:35.731352 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.102262 18293 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.773737 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:36.106467 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.003s	sys 0.001s
I20260709 12:17:36.106254 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:35.612731 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.108476 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.109338 18291 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:36.110488 18292 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:35.771091 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.111543 18291 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:36.107013 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.003s	sys 0.001s
I20260709 12:17:36.113412 18295 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:36.117508 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:36.118094 18263 leader_election.cc:336] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.532s (SENT)
I20260709 12:17:36.114797 18296 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:35.758160 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.118928 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:36.120251 18298 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:36.123888 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:36.120277 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.130241 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:36.131677 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.012s	sys 0.016s
W20260709 12:17:36.132115 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.012s	sys 0.016s
I20260709 12:17:36.132903 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 483) took 1441 ms. Trace:
W20260709 12:17:36.133996 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 807004us
I20260709 12:17:36.134852 18196 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:36.144491 18441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.076s	user 0.024s	sys 0.013s
W20260709 12:17:36.155254 18441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.025s	sys 0.013s
I20260709 12:17:36.156710 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 575) took 1090 ms. Trace:
I20260709 12:17:36.157140 18441 rpcz_store.cc:276] 0709 12:17:35.065826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.066940 (+  1114us) service_pool.cc:224] Handling call
0709 12:17:35.067202 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.067406 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:35.067710 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:17:36.156029 (+1088319us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:36.156140 (+   111us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:36.156258 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.156370 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.156632 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":476,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":1087682,"threads_started":1,"wal-append.queue_time_us":995926}
I20260709 12:17:36.145334 18442 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:36.135783 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.621s (SENT) [suppressed 45 similar messages]
I20260709 12:17:36.133633 18373 rpcz_store.cc:276] 0709 12:17:34.691037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.752817 (+ 61780us) service_pool.cc:224] Handling call
0709 12:17:34.773115 (+ 20298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.843414 (+ 70299us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:35.129608 (+286194us) log.cc:844] Serialized 14 byte log entry
0709 12:17:35.561408 (+431800us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.561982 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.562147 (+   165us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:35.562205 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.562296 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:36.132030 (+569734us) raft_consensus.cc:1685] finished
0709 12:17:36.132109 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.132482 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28642,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":23707,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":12343,"thread_start_us":681397,"threads_started":3,"wal-append.queue_time_us":817017}
I20260709 12:17:36.163857 20847 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.165223 20847 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:36.165602 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.783919 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:36.169546 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 385527us
I20260709 12:17:36.171823 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.174094 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 578) took 1107 ms. Trace:
I20260709 12:17:36.179558 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.180092 18449 rpcz_store.cc:276] 0709 12:17:35.066659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.068449 (+  1790us) service_pool.cc:224] Handling call
0709 12:17:35.068698 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.871128 (+802430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.173516 (+302388us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54001a6154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:36.173851 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844128512}
I20260709 12:17:36.181409 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1690) took 1498 ms. Trace:
I20260709 12:17:36.166379 20847 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.181854 18374 rpcz_store.cc:276] 0709 12:17:34.683177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.689580 (+  6403us) service_pool.cc:224] Handling call
0709 12:17:34.726194 (+ 36614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.690460 (+964266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.180885 (+490425us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b540034ab54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:36.181216 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215949696}
I20260709 12:17:36.184346 20847 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.179193 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.185963 18443 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:36.188077 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.196223 18449 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:17:36.196820 18449 pending_rounds.cc:85] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 3
I20260709 12:17:36.196756 20516 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:36.197367 18449 pending_rounds.cc:107] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:17:36.197928 18449 raft_consensus.cc:2889] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.198671 18442 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:36.200428 20872 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.201062 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.201299 20872 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.202342 18445 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.203620 20898 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:36.204452 20898 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.207864 20967 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:36.209962 20967 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:36.210489 20967 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.210716 21018 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:36.210973 20967 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.212518 21018 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.206926 20898 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.215452 21018 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.210954 20973 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.217150 20973 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.222795 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.206879 20872 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.212167 20924 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:36.147027 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:36.227094 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 863289us
I20260709 12:17:36.231196 18443 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:36.233867 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.239285 18368 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.238550 18293 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:36.163892 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.238233 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102646us
I20260709 12:17:36.245244 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.246140 18371 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:36.247816 18441 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:36.247714 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.249222 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.250115 18441 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:36.250362 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.250562 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.250715 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.250996 18445 raft_consensus.cc:2376] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6. Re-sending same reply.
I20260709 12:17:36.251194 18366 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.251556 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.252022 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.252918 18445 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.251006 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.252323 18449 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:36.241894 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.243383 18448 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:36.255894 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.247006 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 713) took 1044 ms. Trace:
I20260709 12:17:36.258138 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.258373 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:36.259202 18375 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:36.260793 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.261147 18367 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:36.261816 18369 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.259209 19037 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:36.266166 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.266677 18445 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.266927 18367 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.256596 19040 rpcz_store.cc:276] 0709 12:17:35.202880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.203655 (+   775us) service_pool.cc:224] Handling call
0709 12:17:36.246538 (+1042883us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:36.246791 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529879808}
I20260709 12:17:36.271301 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.266888 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:36.276604 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.279466 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.280676 20924 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.281896 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.283155 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.283843 18375 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.281246 19038 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:36.283947 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.285910 19041 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:36.286069 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.281275 20924 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.286643 18372 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.288442 20924 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.290339 21027 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:36.294368 18449 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:36.297333 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.297806 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:36.299889 18263 leader_election.cc:400] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:36.300246 18263 leader_election.cc:403] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:36.300071 18299 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.309377 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.316339 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 700) took 1978 ms. Trace:
I20260709 12:17:36.342193 18372 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:36.326002 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1648) took 1246 ms. Trace:
I20260709 12:17:36.305817 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.322052 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.307969 18449 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:36.317301 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:36.316020 21054 raft_consensus.cc:2764] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:36.326241 21072 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:17:36.318677 21052 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:17:36.306707 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 775) took 1316 ms. Trace:
I20260709 12:17:36.339043 18370 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:36.350520 18366 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:36.315307 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1642) took 1238 ms. Trace:
I20260709 12:17:36.351850 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.343271 18440 rpcz_store.cc:276] 0709 12:17:35.079419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.157474 (+ 78055us) service_pool.cc:224] Handling call
0709 12:17:36.325595 (+1168121us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:36.325821 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937890176}
I20260709 12:17:36.351320 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.306171 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.355369 18369 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:36.354072 18372 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:36.357076 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.308631 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:36.330104 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:36.357918 18445 raft_consensus.cc:2376] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
I20260709 12:17:36.351684 18444 rpcz_store.cc:276] 0709 12:17:35.076641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.076980 (+   339us) service_pool.cc:224] Handling call
0709 12:17:35.077188 (+   208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.891741 (+814553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.314913 (+423172us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54002594d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:36.315129 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1871913728}
W20260709 12:17:36.302825 21119 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.292s	user 0.003s	sys 0.002s
W20260709 12:17:36.304461 21118 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.295s	user 0.006s	sys 0.000s
I20260709 12:17:36.314661 20998 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.360033 18367 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:36.361132 18368 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:36.361811 20937 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.362118 21064 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.362908 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1645) took 1284 ms. Trace:
I20260709 12:17:36.362550 20937 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.363287 18447 rpcz_store.cc:276] 0709 12:17:35.078501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.122567 (+ 44066us) service_pool.cc:224] Handling call
0709 12:17:36.359885 (+1237318us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54001a5c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:36.362749 (+  2864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014276864}
W20260709 12:17:36.364745 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.365290 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.349010 19043 rpcz_store.cc:276] 0709 12:17:34.990357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.990482 (+   125us) service_pool.cc:224] Handling call
0709 12:17:34.990801 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:36.058957 (+1068156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.306251 (+247294us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004a56d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:36.306506 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454726528}
I20260709 12:17:36.366699 18441 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.367530 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.367627 20995 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.367797 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.368451 18370 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.318990 21078 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:17:36.313088 21026 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.347982 21072 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:36.326416 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 714) took 1121 ms. Trace:
I20260709 12:17:36.325397 21049 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.375197 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.377313 18442 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.378551 19038 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:36.330713 21041 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:36.362560 21064 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:36.380980 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.386149 18293 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:36.386688 21064 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.387929 18375 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:36.388283 21064 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.389403 18375 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:36.390661 18375 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.365311 20937 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.392547 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.381558 18297 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:36.382009 18299 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:36.380491 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.376180 21049 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:36.374904 21072 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:36.374492 19036 rpcz_store.cc:276] 0709 12:17:35.204974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.205836 (+   862us) service_pool.cc:224] Handling call
0709 12:17:36.325999 (+1120163us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540060fed4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:36.326244 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237773184}
I20260709 12:17:36.318323 21114 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.372113 20998 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:36.395805 18370 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:36.396073 18368 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:36.397854 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:36.402699 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 94250us
I20260709 12:17:36.411592 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:36.412377 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 95242.8us
I20260709 12:17:36.394997 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.338420 19039 rpcz_store.cc:276] 0709 12:17:34.338012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.338143 (+   131us) service_pool.cc:224] Handling call
0709 12:17:34.352010 (+ 13867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.744492 (+392482us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:34.772788 (+ 28296us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.858238 (+ 85450us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18002c1040. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:17:35.169160 (+310922us) log.cc:844] Serialized 14 byte log entry
0709 12:17:35.315215 (+146055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:35.315335 (+   120us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:35.315402 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.523950 (+208548us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b18002c1040. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:36.315839 (+791889us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54003a0554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:36.315902 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.316124 (+   222us) inbound_call.cc:177] Queueing success response
0709 12:17:36.325281 (+  9157us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b18002c1040. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
Metrics: {"mutex_wait_us":10420,"prepare.queue_time_us":35465,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":190211,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":43276,"spinlock_wait_cycles":941971072,"thread_start_us":437434,"threads_started":2,"wal-append.queue_time_us":526199}
I20260709 12:17:36.408170 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.414036 18367 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:36.403704 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 124845us
I20260709 12:17:36.396564 21054 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.429543 18296 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.429768 18368 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.431643 18373 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:36.430830 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.443588 18297 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:36.444029 18263 leader_election.cc:400] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4.
I20260709 12:17:36.444507 18263 leader_election.cc:403] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:36.444450 18375 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:36.447973 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:36.449685 18296 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:36.420379 21078 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:17:36.442423 20937 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:36.462764 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.466751 18298 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.447346 20967 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.469317 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.456074 20847 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4.
I20260709 12:17:36.464918 20509 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:36.476300 20509 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:36.477357 20937 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:36.477459 20509 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:36.472102 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46830.1us
W20260709 12:17:36.474929 18294 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.008s
I20260709 12:17:36.470229 20980 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:36.475548 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2014) took 1403 ms. Trace:
I20260709 12:17:36.485497 18292 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.470851 20967 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.475585 21052 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:36.486001 18294 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:36.497364 18366 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:36.500842 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1624) took 1015 ms. Trace:
I20260709 12:17:36.487867 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:36.505112 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63611.1us
I20260709 12:17:36.501375 18294 rpcz_store.cc:276] 0709 12:17:35.485160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.577822 (+ 92662us) service_pool.cc:224] Handling call
0709 12:17:36.494938 (+917116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:36.507910 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.512496 21041 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.513070 21041 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.513775 21065 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:36.481637 18300 rpcz_store.cc:276] 0709 12:17:35.038101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.038660 (+   559us) service_pool.cc:224] Handling call
0709 12:17:35.038940 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:35.039432 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.039476 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:35.040217 (+   741us) log.cc:844] Serialized 52 byte log entry
0709 12:17:35.223718 (+183501us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:36.235612 (+1011894us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b1800413980. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:36.291986 (+ 56374us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800413980. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:17:36.402145 (+110159us) log.cc:844] Serialized 14 byte log entry
0709 12:17:36.427815 (+ 25670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.441540 (+ 13725us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:36.441602 (+    62us) raft_consensus.cc:1685] finished
0709 12:17:36.441623 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:36.441771 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46341,"prepare.queue_time_us":176028,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":189408,"raft.run_cpu_time_us":495,"raft.run_wall_time_us":80861,"spinlock_wait_cycles":31928320,"thread_start_us":293072,"threads_started":2,"wal-append.queue_time_us":290847}
W20260709 12:17:36.528004 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.528510 20914 raft_consensus.cc:2764] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:36.471692 20998 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.474932 21072 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.545817 18298 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.492379 20980 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:36.563375 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.494709 21052 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:36.577981 20509 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.584275 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:36.590932 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:36.475437 21049 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.594884 18366 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:36.637224 18366 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:36.595846 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:36.598399 18446 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:17:36.611732 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.607077 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 170185us
I20260709 12:17:36.614741 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.615034 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.614154 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.478652 20937 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.619148 18372 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:36.642872 18370 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:36.643512 18294 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:36.618006 21065 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:36.644908 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60799.9us
W20260709 12:17:36.648551 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.648085 20704 raft_consensus.cc:2764] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:36.655536 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.646313 21114 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:36.655373 20872 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:36.650270 21065 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.651588 20998 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.664054 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.667193 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.658732 18366 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.666150 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:36.672269 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.675606 18293 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:36.678181 19039 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.676651 18300 raft_consensus.cc:2376] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4. Re-sending same reply.
I20260709 12:17:36.680528 20980 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.681748 18375 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:36.683550 20982 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:36.684374 20982 consensus_queue.cc:1243] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:36.684283 20956 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:36.685159 20956 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:17:36.686544 20516 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:36.686915 20516 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:36.665498 20995 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:36.687500 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.687667 20995 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.687871 20516 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.688620 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.689152 20516 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.689759 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.690066 20995 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.690374 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:36.690596 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.690953 18366 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.691210 18300 raft_consensus.cc:2376] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
I20260709 12:17:36.691676 21052 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.693091 21052 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.702349 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:36.702565 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48844.2us
I20260709 12:17:36.711058 18297 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:36.714574 21067 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:17:36.715404 21072 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.671357 20946 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:17:36.676244 21054 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.720317 21054 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.719877 21074 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.722848 21074 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:36.723325 21074 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.669322 20872 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.735267 18291 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:36.726888 18366 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:36.736207 18366 pending_rounds.cc:85] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:17:36.736519 18366 pending_rounds.cc:107] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:36.736804 18366 raft_consensus.cc:2889] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.727377 18298 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:36.729259 21130 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 12:17:36.731215 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:36.732384 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet e9decd1c7d7f475b8890a8519847dca5: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } }
I20260709 12:17:36.734615 20956 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:36.742563 20946 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:36.743026 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67838.6us
I20260709 12:17:36.743732 20872 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.744483 20814 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:36.745460 21000 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:36.781194 21000 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:36.782313 21000 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.783722 21000 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.755690 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.756400 21066 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.786697 21066 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.788899 21066 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:36.757228 20973 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:36.758414 19044 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.760006 19043 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:36.760699 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.761488 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:36.762245 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:36.767963 18367 pending_rounds.cc:85] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:17:36.796852 18367 pending_rounds.cc:107] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:36.796677 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.797423 18367 raft_consensus.cc:2889] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.779099 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:36.790120 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 77536.9us
I20260709 12:17:36.799923 18297 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:36.801963 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 155284us
W20260709 12:17:36.795001 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 105106us
I20260709 12:17:36.805477 18447 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:36.806954 18440 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.813616 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:36.813920 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.748914 20946 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:36.824280 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.798820 20973 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:36.844991 20973 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:36.846834 21133 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:36.798372 20704 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:17:36.847177 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.847837 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.823536 21067 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:17:36.848448 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.826833 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.850257 18297 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:36.851554 18297 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:36.852773 18297 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:36.852685 21054 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.855340 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:36.837240 18412 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.984s (SENT)
W20260709 12:17:36.857573 18412 leader_election.cc:336] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.017s (SENT)
I20260709 12:17:36.839823 20956 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.857829 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:36.841912 21052 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:36.842715 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 111599us
I20260709 12:17:36.843662 20898 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.848383 18440 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:36.849632 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.857391 20872 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:36.841637 20946 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.867784 19041 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.832504 18299 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:36.870121 19037 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:36.870376 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.867851 19036 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:36.873018 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.876345 18443 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.863582 20898 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:36.881700 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:36.883972 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.884948 18448 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:36.878995 21052 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:36.905043 18440 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:36.885994 20898 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.913451 19044 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:36.916604 18440 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:36.903868 21066 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:36.923767 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.929097 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:36.935264 20657 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.935402 20946 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.935842 20657 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.936312 18448 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:36.938689 20657 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:36.943276 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.943547 18412 leader_election.cc:336] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.227s (SENT)
W20260709 12:17:36.944830 18412 leader_election.cc:336] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.242s (SENT)
I20260709 12:17:36.945297 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:36.945920 21041 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.946290 21026 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:36.922992 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 65257.9us
I20260709 12:17:36.949741 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:36.950485 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.959973 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.962391 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.963510 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.963815 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:36.964170 18447 raft_consensus.cc:2376] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4. Re-sending same reply.
I20260709 12:17:36.964593 18442 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.965417 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.966027 18444 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.966406 18449 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:36.966776 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.967553 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:36.967705 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.968365 18296 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:36.968365 18445 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:36.969998 18445 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:36.970697 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:36.971740 18441 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:36.972502 18369 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:36.972756 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:36.964593 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.972821 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.973688 20998 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:36.977016 20998 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:36.977742 18444 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:36.977933 20998 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:36.978502 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:36.973829 18294 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:36.980120 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.982288 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:36.973662 18445 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:36.982900 18366 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:36.974362 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:36.984172 21066 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:36.984266 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:36.984593 21066 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:36.985014 21066 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:36.985159 21041 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:36.985641 21041 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.974635 18442 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:36.989013 19039 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:36.986011 21041 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:36.974535 20872 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.991822 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:36.992766 20872 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:36.993225 20872 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:36.995012 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50252.1us
I20260709 12:17:36.971653 20980 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:36.996913 20980 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:36.998697 20657 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.999214 20657 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.000003 21026 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:37.001514 20657 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.002771 19041 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:36.958844 21065 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.003515 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.003630 18443 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:37.003939 20980 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:36.994148 20872 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.009071 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:37.012494 18446 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:37.010058 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60461.3us
I20260709 12:17:37.010603 20704 raft_consensus.cc:2764] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:17:37.013923 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.024245 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.029790 18448 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:37.030681 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:37.033434 18300 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.033748 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.033912 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.033887 18440 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:37.035372 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.022382 21074 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.038230 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.030489 20980 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.985981 20898 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.032287 21026 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.043754 18443 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:37.040803 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.046692 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.049723 18442 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:37.052162 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:37.054639 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:37.060747 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57355.8us
I20260709 12:17:37.063261 21074 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.054232 21066 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.039978 20704 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:17:37.065331 21066 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:37.071655 19038 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:17:37.114454 18440 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:37.115403 18443 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:37.118719 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.119746 18443 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:37.052500 21072 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.128400 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.371140 21065 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:37.382660 18296 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
W20260709 12:17:37.394555 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 342435us
I20260709 12:17:37.395442 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:37.413254 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.456982 21064 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:37.457396 21064 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.458146 21064 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.459131 21064 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:37.474704 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.131137 20704 consensus_queue.cc:1243] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:37.507083 18300 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
W20260709 12:17:37.140331 21123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.021s	sys 0.030s
W20260709 12:17:37.511608 21123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.025s	sys 0.030s
I20260709 12:17:37.515844 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.516484 20914 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.520807 21114 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:37.520853 20914 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:37.521442 20914 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.521770 21052 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.522202 21052 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.522526 21052 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.521459 21114 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.523694 21026 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.524225 21026 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.525347 21114 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.526180 21026 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:37.526714 21026 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:37.517256 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 121906us
I20260709 12:17:37.528723 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:37.529976 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:37.531500 21064 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.531728 21114 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:37.124051 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.532040 21064 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:37.532577 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 408577us
I20260709 12:17:37.532814 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.533098 21064 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.533865 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:37.534247 21114 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:37.527123 21026 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.519757 21053 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.538975 20967 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:37.540041 21053 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:37.540410 21053 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.540860 21065 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.541270 21065 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:37.541764 21065 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.541271 21053 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.536580 21078 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:17:37.545964 21078 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:37.538533 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:37.546379 21078 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.547266 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.550884 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.553756 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.552572 21052 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.555768 21052 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.548580 21026 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:37.556347 21052 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.556697 21026 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.547452 21078 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.569237 21026 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.569644 21078 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.570158 21026 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.557098 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.557878 21102 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:37.572638 21102 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.518680 21158 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:37.132428 21072 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.574862 21072 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.559031 20914 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:37.560216 21128 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.965s	user 0.009s	sys 0.000s
I20260709 12:17:37.578876 20914 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:37.564834 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 513265us
I20260709 12:17:37.133189 21066 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.434s
I20260709 12:17:37.586905 20872 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:37.587352 21065 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.589946 21065 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:37.590479 20973 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:37.594266 20946 consensus_meta.cc:287] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.003s
W20260709 12:17:37.605093 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.605453 21065 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.612288 21065 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:37.607867 20973 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:37.632264 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 81442.7us
I20260709 12:17:37.580626 21052 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.599073 21114 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.643488 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:37.651418 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 117490us
I20260709 12:17:37.652364 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.647768 21151 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:17:37.661409 20898 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:37.665256 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 45530.6us
I20260709 12:17:37.666388 20898 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:37.667101 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.667443 21114 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.668110 21078 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:37.668366 21064 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:37.668901 21064 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.669473 21078 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:37.669781 21064 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.670468 20914 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:37.670861 20914 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.671275 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.671521 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.667697 20898 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:37.672075 18298 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:37.671598 20914 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.672873 20914 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:37.673434 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.674177 18300 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:37.673969 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.674816 20898 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.675329 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.676364 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.676730 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:37.676972 18446 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.003s
I20260709 12:17:37.677271 18295 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:37.677556 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.678200 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.677284 18446 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:37.682242 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.683091 21160 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.683485 18369 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:37.684257 18297 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:17:37.684253 18300 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:37.689970 18295 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:37.690937 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.696585 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:37.696328 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 140173us
I20260709 12:17:37.704392 18369 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:37.705470 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.711469 18374 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:37.656059 21052 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.714459 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:37.715039 21163 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.716090 18298 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:37.719736 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:37.719782 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.720603 18369 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:37.704025 21044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.014s	sys 0.020s
I20260709 12:17:37.722702 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:37.722949 21163 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.723439 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.724812 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:37.724371 21163 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.725797 18370 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.709563 21064 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
W20260709 12:17:37.727499 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.728935 20956 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:37.721668 21044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.014s	sys 0.021s
I20260709 12:17:37.730054 21159 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.730595 21159 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.731709 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:37.732748 20980 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:37.732748 21159 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.733287 20980 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:37.734190 20980 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.744254 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.746810 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.754110 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:37.751163 21044 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.759502 21064 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:17:37.769061 21160 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:37.769446 21160 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:37.769809 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.770347 21160 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.752836 21159 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.771476 21159 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:37.771978 21159 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.772540 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:37.775322 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:37.776029 18440 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:37.776952 21173 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:17:37.777758 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.778517 18445 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:37.786422 18440 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:37.771499 21160 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:37.798146 18293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.008s	sys 0.011s
I20260709 12:17:37.750779 21114 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:17:37.798534 18293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.010s	sys 0.012s
W20260709 12:17:37.798475 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54709.3us
W20260709 12:17:37.830155 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55410.6us
I20260709 12:17:37.834051 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.835580 18440 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:37.836634 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 86235.9us
I20260709 12:17:37.833236 21151 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:37.839186 20914 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.839541 20924 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.839772 20914 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.840272 20924 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.839505 21151 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.831270 21054 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:37.843930 20924 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.842433 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.851143 21114 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:37.852304 21114 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.852671 21151 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.850728 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.852679 21114 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:37.854342 21041 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.864s	user 0.011s	sys 0.000s
I20260709 12:17:37.854537 20914 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.833508 21044 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.857343 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.855680 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:37.858660 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:37.859678 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:37.862937 18369 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:37.862286 21163 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:37.854759 21041 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.873867 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.858003 21064 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:37.875478 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.877480 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:37.879742 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:37.880425 18294 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:37.881065 21164 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:37.881582 21164 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.882450 21164 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:37.883422 21064 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.885573 21064 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:37.887987 21159 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.890266 21159 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:37.863425 21151 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:37.891161 21044 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.891726 21151 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:37.893021 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.895862 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:37.896414 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:37.897403 18445 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:37.867368 20967 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.898638 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:37.892683 21151 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:37.903476 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.904156 18445 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:37.878870 21054 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:37.860847 21168 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:17:37.875205 21169 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:37.903746 20967 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.901695 21163 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.893539 20937 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:17:37.908941 21054 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.910521 21026 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.923804 20967 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.915475 21066 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:37.933463 21054 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:37.924455 20924 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:17:37.944439 21026 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.953792 21026 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:37.954947 21177 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:17:37.959770 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:37.959525 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.961519 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:37.961427 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:37.945338 21066 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:37.965274 21185 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:37.965724 21185 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.965569 21183 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.967415 21052 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.967936 21052 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:37.968750 21052 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:37.970249 21053 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.970496 21064 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:37.971370 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.971077 21064 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.973234 21064 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:37.974102 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.034s	sys 0.031s
W20260709 12:17:37.974452 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.034s	sys 0.031s
W20260709 12:17:37.974695 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 947037us
I20260709 12:17:37.975346 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.977078 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:37.979122 21167 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.979708 21167 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:37.980669 21167 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:37.982506 21049 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.966629 21185 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:37.985275 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.985710 21185 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.985186 21074 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:37.988397 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:37.991133 18339 connection.cc:256] RPC call timeout handler was delayed by -1.1366232670000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:17:37.998263 19038 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 781) took 1239 ms (client timeout 1504 ms). Trace:
W20260709 12:17:37.999975 19038 rpcz_store.cc:269] 0709 12:17:36.758434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.761223 (+  2789us) service_pool.cc:224] Handling call
0709 12:17:37.998096 (+1236873us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540020bcd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:37.998217 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632442624}
I20260709 12:17:38.002575 20937 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.990765 21151 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:38.002913 21114 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:38.002750 21179 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:17:38.003288 21151 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:38.003391 21178 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:17:37.995178 20998 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.012835 20998 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.014910 20998 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.012256 18300 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:38.018955 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:38.019546 18300 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:38.019973 21041 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.007076 21151 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:38.021011 21066 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.003201 20937 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:38.032969 20937 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.033959 21175 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:17:38.037889 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 909542us
I20260709 12:17:38.031832 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:38.040232 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.040920 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.043582 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:38.044881 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.048640 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.062521 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:38.064692 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:38.065492 18440 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:38.035758 20980 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.069687 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:38.076472 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:38.079977 18366 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:38.080430 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.081231 18370 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 4: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:38.081310 21161 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.397s	user 0.000s	sys 0.003s
I20260709 12:17:38.082090 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:38.082365 21153 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:17:38.083504 21186 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:38.083932 21186 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:38.084483 18448 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:38.084976 21186 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.085558 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:38.085670 21129 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.501s	user 0.005s	sys 0.005s
I20260709 12:17:38.086251 18443 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:38.088271 19039 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:38.088464 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1703) took 1761 ms. Trace:
I20260709 12:17:38.088792 18371 rpcz_store.cc:276] 0709 12:17:36.326825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.344053 (+ 17228us) service_pool.cc:224] Handling call
0709 12:17:36.369611 (+ 25558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:36.370167 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:36.370228 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:36.370904 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:17:36.456165 (+ 85261us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:36.456204 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:36.456271 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.059705 (+1603434us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:38.088179 (+ 28474us) raft_consensus.cc:1685] finished
0709 12:17:38.088201 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.088405 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4002,"raft.queue_time_us":1530,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":49411328,"thread_start_us":84746,"threads_started":1,"wal-append.queue_time_us":116164}
I20260709 12:17:38.091172 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:38.045159 20924 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.077049 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:38.021116 21054 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.048401 20995 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.051154 21066 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.100394 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:38.047370 20998 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:38.100036 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 988982us
I20260709 12:17:38.097373 19038 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:38.101197 18446 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:38.099423 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 147859us
I20260709 12:17:38.096477 21041 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.103981 18370 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:38.108387 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:38.112124 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63745.2us
I20260709 12:17:38.110067 18366 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:17:38.113909 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 152415us
W20260709 12:17:38.114563 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.069228 20516 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.112371 20998 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.118261 20998 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.119369 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.086264 21186 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.100471 20924 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.125106 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:38.110939 20657 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.111111 20956 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.022871 21163 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.115329 21169 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.111114 21041 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:38.155511 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:38.162148 19038 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:38.128943 20924 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.171262 19041 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:38.151134 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800038720 after lost signal to thread 17160
W20260709 12:17:38.204478 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08000386a0 after lost signal to thread 18232
W20260709 12:17:38.206061 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800039120 after lost signal to thread 18324
W20260709 12:17:38.159881 18339 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.621s (SENT)
W20260709 12:17:38.208137 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48277.2us
W20260709 12:17:38.209846 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9080 after lost signal to thread 18399
W20260709 12:17:38.210314 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08002d94c0 after lost signal to thread 18400
I20260709 12:17:38.166671 21041 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.214190 18374 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:38.218433 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:38.164502 21114 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.125633 20516 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.227639 18339 leader_election.cc:336] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.780s (SENT)
W20260709 12:17:38.214175 21175 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:38.185756 21167 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.124377 21196 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.215211 21026 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:38.163620 20956 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.155174 20657 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.272564 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44979.3us
I20260709 12:17:38.293296 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.293502 19043 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:17:38.237440 21195 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.271936 18374 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:38.311789 19044 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:38.222872 20814 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:17:38.314461 21175 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:17:38.352859 20898 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.353566 21169 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:38.355482 21067 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:38.377471 18339 leader_election.cc:336] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.675s (SENT)
I20260709 12:17:38.359706 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:38.269363 21193 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:17:38.310703 20973 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:17:38.388784 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44125.1us
W20260709 12:17:38.394230 18291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 454) took 1723 ms (client timeout 2005 ms). Trace:
I20260709 12:17:38.394972 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.270634 21196 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.389439 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 270113us
I20260709 12:17:38.398703 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:38.394876 18291 rpcz_store.cc:269] 0709 12:17:36.664445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.664628 (+   183us) service_pool.cc:224] Handling call
0709 12:17:38.387484 (+1722856us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b54000aa0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:38.387705 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142318848}
I20260709 12:17:38.345221 20946 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:17:38.406325 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:38.314802 21195 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.385890 21054 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.393291 20898 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.532366 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 126126us
I20260709 12:17:38.533776 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:38.535115 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 136477us
I20260709 12:17:38.416852 21049 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:38.538144 21197 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 12:17:38.540014 21205 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:17:38.542109 21200 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 12:17:38.543453 21198 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
W20260709 12:17:38.548401 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.504977 21067 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:17:38.550212 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.415391 21208 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.556306 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:38.503635 20814 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
W20260709 12:17:38.584164 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50471.8us
I20260709 12:17:38.504262 20967 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.550809 21167 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:38.547715 21052 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.585726 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:38.586639 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 221158us
I20260709 12:17:38.588480 21095 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:38.549289 20973 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:38.599462 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:38.603269 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.552130 21067 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:38.554267 21049 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:38.558534 20516 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.623206 18293 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:17:38.626114 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.013s
W20260709 12:17:38.626617 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.000s	sys 0.014s
W20260709 12:17:38.626965 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 522395us
W20260709 12:17:38.629664 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 72461.9us
I20260709 12:17:38.621389 20509 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:38.636924 21202 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:17:38.625443 20956 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:38.629596 20516 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:38.638399 20898 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.619701 21208 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.657697 21208 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:38.659654 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800244780 after lost signal to thread 18473
W20260709 12:17:38.660010 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800045580 after lost signal to thread 18474
W20260709 12:17:38.661166 18229 debug-util.cc:398] Leaking SignalData structure 0x7b080035cb20 after lost signal to thread 19068
W20260709 12:17:38.661808 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800389ce0 after lost signal to thread 21027
W20260709 12:17:38.662285 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08002209e0 after lost signal to thread 21087
W20260709 12:17:38.662637 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08002665a0 after lost signal to thread 21104
W20260709 12:17:38.662672 21186 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.000s
I20260709 12:17:38.663450 20516 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:38.664264 21217 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:38.664439 20516 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.665725 21066 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.001s
W20260709 12:17:38.666905 21151 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 12:17:38.667346 21151 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.667654 21227 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.639369 21204 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.640000 18371 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:38.640509 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.641357 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:38.641227 21049 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.642082 21212 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.552237 21169 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.638854 21209 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.645190 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:38.644752 21026 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:38.648692 21207 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:17:38.686679 21196 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.650938 20967 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.687083 21220 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.646739 21221 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:17:38.652045 21195 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.622073 21052 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.670352 18369 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:38.707723 21220 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:38.663079 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecd40 after lost signal to thread 21111
W20260709 12:17:38.677146 21185 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.000s
I20260709 12:17:38.689139 20973 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:38.709417 20973 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:38.709549 21052 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.710338 20973 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.682451 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.712252 18370 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:38.684772 21212 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:38.688609 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43518.4us
W20260709 12:17:38.704499 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63195.7us
I20260709 12:17:38.719995 18445 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:38.644274 21224 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.646484 20509 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.694772 21026 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:38.648846 21067 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.688098 21209 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.708359 21220 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.680451 21204 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.728012 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:38.730976 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.750752 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:38.752125 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet c1f155085cb647ff957b468384227872: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:17:38.759076 18339 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.069s (SENT)
I20260709 12:17:38.783629 18442 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:38.687915 21151 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:38.783660 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:38.787711 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56854.9us
I20260709 12:17:38.714182 21212 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:38.788677 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.752125 21224 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:38.781852 20998 consensus_meta.cc:287] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.002s
W20260709 12:17:38.847164 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 290938us
I20260709 12:17:38.812821 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.819993 18372 raft_consensus.cc:2410] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:38.839337 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:38.805750 21209 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.809446 21211 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.864164 21204 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.864025 21211 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.812004 20704 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.795161 21224 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.815364 21151 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:17:38.862543 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 147220us
W20260709 12:17:38.822119 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.852547 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:38.905938 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 53489.9us
I20260709 12:17:38.905998 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:38.820971 20980 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.723s	user 0.007s	sys 0.003s
W20260709 12:17:38.813947 20924 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.001s
I20260709 12:17:38.911202 18375 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:38.861579 21227 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:38.914379 18375 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:38.803818 21237 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.914893 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.821614 21067 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.919867 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:38.915764 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67835.3us
I20260709 12:17:38.793213 21222 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.921458 18371 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:38.866462 21153 raft_consensus.cc:2764] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:38.792666 21000 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:17:38.923030 18375 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:38.791556 21026 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.924515 20956 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:38.924715 21219 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.924517 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:38.907971 21209 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.927574 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.927908 21241 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.811753 20998 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.931428 21210 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:17:38.956804 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.932881 21211 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:38.933269 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.917124 21227 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.945305 20956 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.954545 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:38.955250 21041 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.001s
I20260709 12:17:38.955852 20509 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.952003 21241 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:38.945477 21219 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:38.946065 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:38.960928 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:38.976877 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.977392 18371 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:38.916762 20924 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:38.966797 20956 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:38.961861 21227 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.968405 21211 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.999505 21095 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:39.000263 21095 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.970270 21041 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.015839 20509 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.016351 21224 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.016896 21224 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.017237 21041 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.018304 21220 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.018962 21224 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.019549 21224 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.020033 21224 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.020668 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:39.021806 21049 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:39.022018 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet ea246fa8cc7e4edbb136da2ee92fb1d3: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:39.022171 21049 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.022279 21224 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.022629 21049 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.945184 21237 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.026804 21237 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:38.975551 21241 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.016402 20509 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:38.978693 21219 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.031312 20509 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:39.032503 21190 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.868s	user 0.008s	sys 0.029s
I20260709 12:17:39.032789 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1674) took 1259 ms. Trace:
I20260709 12:17:39.033170 18444 rpcz_store.cc:276] 0709 12:17:37.773423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.656585 (+883162us) service_pool.cc:224] Handling call
0709 12:17:38.656929 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.713203 (+ 56274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.713237 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:38.713817 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:17:38.945567 (+231750us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:38.945633 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.023636 (+ 78003us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54000d8c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:39.032396 (+  8760us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.032459 (+    63us) raft_consensus.cc:1685] finished
0709 12:17:39.032475 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.032704 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3714,"raft.queue_time_us":1154,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":28118,"spinlock_wait_cycles":128615808,"thread_start_us":231290,"threads_started":1,"wal-append.queue_time_us":275152}
I20260709 12:17:38.975776 21163 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:17:39.035079 21230 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:39.035549 21230 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:39.018980 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:39.041575 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 87154.1us
I20260709 12:17:38.957388 21209 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.044893 21209 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.046696 21114 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.047214 21114 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.049325 21114 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.050180 21095 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.051481 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
I20260709 12:17:39.052304 18412 leader_election.cc:403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:39.059571 21216 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.060448 18367 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:39.060103 21216 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.060853 21095 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.062111 21065 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
I20260709 12:17:39.062652 20509 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:39.062965 21216 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.063186 20509 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:38.984521 21236 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.063649 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:39.064455 21236 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.064363 20509 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.065753 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:39.066509 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.067004 18443 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.070611 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:38.984367 21222 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:39.080619 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:39.055060 21188 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.012s	user 0.006s	sys 0.012s
I20260709 12:17:39.082841 18368 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:39.083909 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:39.087898 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:39.056586 21220 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:39.096562 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 139978us
I20260709 12:17:39.054003 20657 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:39.036448 21230 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.066144 20509 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:39.110255 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.082298 21053 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.088840 21065 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:39.138722 18443 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:39.152479 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:39.091270 20814 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:17:39.059087 21227 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.056645 21241 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.151894 21212 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:39.168136 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.155546 21209 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.156399 21185 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:39.177775 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:39.166081 20814 consensus_queue.cc:1243] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:39.178606 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.181022 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:39.183096 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.184078 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.185034 19041 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.184700 18441 raft_consensus.cc:2376] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8. Re-sending same reply.
I20260709 12:17:39.187770 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.188051 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.188354 18447 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.188733 19039 raft_consensus.cc:2368] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:39.191721 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.158576 21053 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.192561 18440 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:39.192811 18449 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.196233 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.235827 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:39.199747 21227 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.199759 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.201524 19039 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:39.204528 21242 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.216s	user 0.002s	sys 0.002s
I20260709 12:17:39.208326 21241 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:39.215761 20924 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:39.216867 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.177030 21209 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.221568 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.223083 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.223423 19038 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:39.225016 21195 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:39.225777 21044 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:39.252737 21044 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:39.234313 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:39.253736 18374 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.254349 18447 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:39.254630 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.256132 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.257337 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.257232 20956 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.186396 20998 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.248302 19038 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:39.260614 19038 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.261755 19038 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:39.262760 21187 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.216s	user 0.017s	sys 0.012s
I20260709 12:17:39.243032 21065 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:17:39.266543 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.267660 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.238966 18443 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:17:39.241849 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.246027 18375 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:39.194836 21216 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.247268 18448 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:39.249294 21247 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:17:39.274873 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:39.275462 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.271210 18443 pending_rounds.cc:85] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 7
I20260709 12:17:39.276721 18443 pending_rounds.cc:107] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:17:39.277086 18443 raft_consensus.cc:2889] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:39.277330 18374 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:39.274302 18449 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:39.273737 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.270236 18368 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:39.193442 21026 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:39.278754 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.271581 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.279600 18371 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:39.192378 21210 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:39.272723 19038 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:39.284515 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:39.286700 18293 raft_consensus.cc:2376] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4. Re-sending same reply.
I20260709 12:17:39.287575 18291 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
I20260709 12:17:39.287482 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:39.291697 18293 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:17:39.288767 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:39.292587 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.290840 18375 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:39.292953 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:39.293337 18448 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.293459 19041 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.296280 18440 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:39.297518 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.292675 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:39.301429 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.287406 21114 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.304320 19042 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.304260 21114 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.305104 21049 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.306583 21114 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.307112 20509 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.307335 20956 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:39.307482 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:39.308194 18368 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:39.308651 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.305424 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.309185 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:39.310002 19040 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:39.269766 21255 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.282639 21163 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.316529 18371 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:39.322121 18371 pending_rounds.cc:85] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 2
I20260709 12:17:39.322456 18371 pending_rounds.cc:107] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:39.322779 18371 raft_consensus.cc:2889] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:39.274557 21064 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:39.282414 21168 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.338913 18299 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.340867 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.307642 20509 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.315318 18448 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:39.281494 21227 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:39.344305 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:39.323173 18371 pending_rounds.cc:107] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:39.324574 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:39.345714 18371 raft_consensus.cc:2889] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:39.345887 18368 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:39.326871 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:39.327196 19042 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:39.317559 21255 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.329931 21209 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.332890 18300 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:39.333842 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.334548 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.335516 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:39.368023 18298 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.336970 18296 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:39.338790 21168 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.317886 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:39.366973 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:39.369378 19038 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:39.347043 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.370851 18299 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:39.350400 18440 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:17:39.357100 18443 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.356372 19041 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:39.374269 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:39.376312 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.282727 21216 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.378002 18448 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:39.380247 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.380482 21168 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.380857 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:39.368750 21064 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.382640 18293 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.384383 21064 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:39.385237 18296 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:17:39.387102 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.387643 21183 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:39.388111 21183 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:39.387979 18296 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:39.391522 21159 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:39.392052 21159 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.392230 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.392999 18299 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:39.394163 21159 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.394572 21053 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.360013 21204 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:39.395804 21204 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.397770 18294 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:39.309218 21237 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:17:39.369302 21241 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:39.398391 18373 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:39.403514 19039 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:39.302117 21210 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:39.410662 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.416018 18441 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:39.313992 21026 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.417354 18291 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:39.375295 21227 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:39.375886 21219 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.369546 21163 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:39.405494 21168 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
W20260709 12:17:39.792171 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.806510 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 607) took 1087 ms. Trace:
W20260709 12:17:40.020681 18447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.029s	sys 0.054s
W20260709 12:17:39.632272 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 294) took 21908 ms (client timeout 15000 ms). Trace:
I20260709 12:17:40.251731 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.519742 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:39.182163 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:40.252367 18447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.031s	sys 0.083s
I20260709 12:17:40.253084 18299 raft_consensus.cc:2410] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:17:40.177824 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.571837 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.551277 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:39.711467 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 397383us
I20260709 12:17:39.748636 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 608) took 1022 ms. Trace:
I20260709 12:17:40.194494 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:39.407264 21236 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:39.611961 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 386104us
I20260709 12:17:39.495668 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:40.262408 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.263736 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:40.271195 18367 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.447374 18293 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:39.524782 18449 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:39.521332 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:40.186046 18298 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.818s	user 0.010s	sys 0.016s
I20260709 12:17:40.275637 18298 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
W20260709 12:17:39.810516 21211 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:17:40.251592 18446 rpcz_store.cc:276] 0709 12:17:38.719380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.719529 (+   149us) service_pool.cc:224] Handling call
0709 12:17:38.751575 (+ 32046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.847801 (+ 96226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.847829 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:38.916996 (+ 69167us) log.cc:844] Serialized 26 byte log entry
0709 12:17:38.983180 (+ 66184us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:39.272178 (+288998us) log.cc:844] Serialized 14 byte log entry
0709 12:17:39.279911 (+  7733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.806013 (+526102us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b54001856d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:39.806085 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.806148 (+    63us) raft_consensus.cc:1685] finished
0709 12:17:39.806259 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.806438 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88614,"prepare.queue_time_us":185425,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":2721,"raft.queue_time_us":27526,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":23774,"spinlock_wait_cycles":220429184,"thread_start_us":204843,"threads_started":2,"wal-append.queue_time_us":113255}
I20260709 12:17:40.253342 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.271720 18448 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:40.278065 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2108) took 1079 ms. Trace:
W20260709 12:17:39.553370 21252 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
W20260709 12:17:40.250396 19036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.000s	sys 0.012s
W20260709 12:17:39.756819 21185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.000s	sys 0.024s
W20260709 12:17:39.887339 21212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.036s	sys 0.032s
I20260709 12:17:40.376617 18444 rpcz_store.cc:276] 0709 12:17:39.186924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.805126 (+618202us) service_pool.cc:224] Handling call
0709 12:17:40.265839 (+460713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:39.488204 21053 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:40.329010 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:39.615132 21222 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.373840 21210 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.406680 20998 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.906s
I20260709 12:17:40.374907 21209 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.647090 18194 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 6, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:39.416103 21114 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:39.488973 21241 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:40.280247 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:40.404618 18371 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:40.285264 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:39.487715 20509 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.281721 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:40.253706 21169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.003s	sys 0.009s
I20260709 12:17:40.379030 18367 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:40.273191 20914 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.382427 18372 raft_consensus.cc:2368] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7.
I20260709 12:17:40.372274 21227 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:40.378234 21185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.000s	sys 0.029s
I20260709 12:17:39.768579 21191 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:39.799649 21239 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:40.388695 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.502732 18444 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:40.167678 20924 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.391662 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1737) took 1108 ms. Trace:
W20260709 12:17:40.427786 21169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.006s	sys 0.012s
I20260709 12:17:40.393606 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.533649 19041 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:40.405306 21209 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.534950 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.535745 19041 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:40.536813 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:40.252141 18196 rpcz_store.cc:269] 0709 12:17:17.723830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.724111 (+   281us) service_pool.cc:224] Handling call
0709 12:17:39.632173 (+21908062us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:36.399465 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:36.476208 (+ 76743us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:36.476270 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:17:36.476326 (+    56us) tablet.cc:662] Decoding operations
0709 12:17:36.477115 (+   789us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:36.477226 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:36.477253 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:36.477763 (+   510us) tablet.cc:701] Row locks acquired
0709 12:17:36.477827 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:17:36.478013 (+   186us) write_op.cc:270] Start()
0709 12:17:36.478175 (+   162us) write_op.cc:276] Timestamp: P: 1783599456477923 usec, L: 0
0709 12:17:36.478210 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:17:36.506465 (+ 28255us) log.cc:844] Serialized 1312 byte log entry
0709 12:17:37.013256 (+506791us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c65d720ac 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b
0709 12:17:37.512892 (+499636us) op_driver.cc:464] REPLICATION: finished
0709 12:17:37.514282 (+  1390us) write_op.cc:301] APPLY: starting
0709 12:17:37.514405 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:37.514558 (+   153us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:37.514593 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:37.515210 (+   617us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:37.515534 (+   324us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:37.515585 (+    51us) write_op.cc:312] APPLY: finished
0709 12:17:37.515890 (+   305us) log.cc:844] Serialized 35 byte log entry
0709 12:17:37.516071 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:37.516169 (+    98us) write_op.cc:454] Released schema lock
0709 12:17:37.516162 (+    -7us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7932a 00007f3c68dfef14 00007f3c68dfeec3 00007f3c68dfee38 00007f3c68dfedf3 00007f3c68dfe0dc 00007f3c68e27e16 00007f3c68e0fcbb 00007f3c68dfa39d 00007f3c68dfdd75
0709 12:17:37.516222 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:37.516627 (+   405us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:37.516653 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1170,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":1102,"apply.run_cpu_time_us":2324,"apply.run_wall_time_us":2332,"num_ops":3,"prepare.queue_time_us":61165,"prepare.run_cpu_time_us":61432,"prepare.run_wall_time_us":1156567,"raft.queue_time_us":11448,"raft.run_cpu_time_us":6315,"raft.run_wall_time_us":319800,"replication_time_us":1034519,"spinlock_wait_cycles":772712704,"thread_start_us":965114,"threads_started":4,"wal-append.queue_time_us":602295,"wal-append.run_cpu_time_us":11226,"wal-append.run_wall_time_us":1350406}]]}
I20260709 12:17:40.537531 19041 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:40.260047 18445 rpcz_store.cc:276] 0709 12:17:38.726220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.726378 (+   158us) service_pool.cc:224] Handling call
0709 12:17:38.726743 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.782486 (+ 55743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.782516 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:38.791621 (+  9105us) log.cc:844] Serialized 26 byte log entry
0709 12:17:39.091862 (+300241us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:39.671775 (+579913us) log.cc:844] Serialized 14 byte log entry
0709 12:17:39.741291 (+ 69516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.741390 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.741446 (+    56us) raft_consensus.cc:1685] finished
0709 12:17:39.741464 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.748560 (+  7096us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79440,"prepare.queue_time_us":99958,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":69162,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":42450,"thread_start_us":667726,"threads_started":2,"wal-append.queue_time_us":253914}
I20260709 12:17:40.539826 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:40.540762 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1684) took 1144 ms. Trace:
I20260709 12:17:40.541208 18445 rpcz_store.cc:276] 0709 12:17:39.395900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.539532 (+1143632us) service_pool.cc:224] Handling call
0709 12:17:40.540694 (+  1162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.542234 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:39.748304 21237 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.543540 21237 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.546114 21237 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.147445 21216 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:39.487954 21159 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.548355 21159 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.549401 21209 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.550128 21209 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:40.329268 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 521) took 1417 ms. Trace:
I20260709 12:17:40.550740 21209 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.553257 21209 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:40.378427 19036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.000s	sys 0.017s
W20260709 12:17:40.382975 21212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.057s	sys 0.058s
I20260709 12:17:40.147800 21255 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.414413 18366 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.410434 19043 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:40.410526 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 125329us
I20260709 12:17:40.416026 21241 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.318765 21236 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.265280 21078 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 12:17:40.415889 18194 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.404119 21114 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.551009 21159 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.458727 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.591823 18448 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:40.499118 21163 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.529172 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2119) took 1104 ms. Trace:
I20260709 12:17:40.593084 18444 rpcz_store.cc:276] 0709 12:17:39.424479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.388438 (+963959us) service_pool.cc:224] Handling call
0709 12:17:40.529076 (+140638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.402263 21210 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.529811 18373 rpcz_store.cc:276] 0709 12:17:39.276491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.276756 (+   265us) service_pool.cc:224] Handling call
0709 12:17:40.384568 (+1107812us) spinlock_profiling.cc:243] Waited 99.2 ms on lock 0x7b540034ab54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:40.384694 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228171520}
I20260709 12:17:40.265077 20956 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.532845 19039 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:40.551409 18369 rpcz_store.cc:276] 0709 12:17:38.911943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.912074 (+   131us) service_pool.cc:224] Handling call
0709 12:17:39.027518 (+115444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.027731 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:39.028034 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:17:39.094332 (+ 66298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:39.094400 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:39.095650 (+  1250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.095720 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.329030 (+1233310us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54003f2d50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:40.329172 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29141,"prepare.queue_time_us":637,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":109,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":6407,"spinlock_wait_cycles":266013696,"thread_start_us":65898,"threads_started":1,"wal-append.queue_time_us":231335,"wal-append.run_cpu_time_us":1062,"wal-append.run_wall_time_us":1006062}
I20260709 12:17:40.596953 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:40.257956 21164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.021s	user 0.016s	sys 0.027s
I20260709 12:17:40.542868 18445 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:40.597565 18444 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:40.395308 21222 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.456030 21193 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:40.567677 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.074s	sys 0.124s
I20260709 12:17:40.430825 20509 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.390396 21053 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.502260 21191 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:40.616665 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.074s	sys 0.124s
W20260709 12:17:40.568570 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.035s	sys 0.053s
W20260709 12:17:40.556602 21252 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:17:40.589061 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 728) took 1268 ms. Trace:
I20260709 12:17:40.620002 19043 rpcz_store.cc:276] 0709 12:17:39.318290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.392150 (+1073860us) service_pool.cc:224] Handling call
0709 12:17:40.410255 (+ 18105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.586215 (+175960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.586410 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.592281 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2120) took 1167 ms. Trace:
I20260709 12:17:40.590312 18194 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.595728 18440 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:40.497496 20914 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.597231 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 559) took 1331 ms. Trace:
I20260709 12:17:40.554055 20924 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.625869 19039 rpcz_store.cc:276] 0709 12:17:39.266233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.411884 (+1145651us) service_pool.cc:224] Handling call
0709 12:17:40.514264 (+102380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.597000 (+ 82736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.597182 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.597765 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2118) took 1187 ms. Trace:
I20260709 12:17:40.598075 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1697) took 1193 ms. Trace:
I20260709 12:17:40.627911 18444 rpcz_store.cc:276] 0709 12:17:39.404475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.596694 (+1192219us) service_pool.cc:224] Handling call
0709 12:17:40.598024 (+  1330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.622645 18448 rpcz_store.cc:276] 0709 12:17:39.424875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.401990 (+977115us) service_pool.cc:224] Handling call
0709 12:17:40.592237 (+190247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.629045 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.629726 18444 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:40.629842 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.630209 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1694) took 1230 ms. Trace:
I20260709 12:17:40.607347 18366 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:40.630451 19043 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
W20260709 12:17:40.619037 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.035s	sys 0.053s
W20260709 12:17:40.604806 21164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.379s	user 0.023s	sys 0.031s
I20260709 12:17:40.590255 21195 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:40.592478 21259 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.637583 18367 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.627836 18445 rpcz_store.cc:276] 0709 12:17:39.410495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.541956 (+1131461us) service_pool.cc:224] Handling call
0709 12:17:40.597697 (+ 55741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.596395 21159 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.603807 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.630535 18444 rpcz_store.cc:276] 0709 12:17:39.399902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.628807 (+1228905us) service_pool.cc:224] Handling call
0709 12:17:40.630135 (+  1328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.633793 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:40.502259 21239 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.634225 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 5 to 6, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:40.639791 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.640265 18298 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:40.641100 18369 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.610298 21114 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.643600 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.662320 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2121) took 1121 ms. Trace:
I20260709 12:17:40.662703 18448 rpcz_store.cc:276] 0709 12:17:39.541250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.629066 (+1087816us) service_pool.cc:224] Handling call
0709 12:17:40.662265 (+ 33199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.662999 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1690) took 1265 ms. Trace:
W20260709 12:17:40.644757 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 393861us
I20260709 12:17:40.663426 18445 rpcz_store.cc:276] 0709 12:17:39.397288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.639559 (+1242271us) service_pool.cc:224] Handling call
0709 12:17:40.662944 (+ 23385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.645418 18294 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:40.664223 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2062) took 1035 ms. Trace:
I20260709 12:17:40.664517 18294 rpcz_store.cc:276] 0709 12:17:39.629009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.630154 (+1001145us) service_pool.cc:224] Handling call
0709 12:17:40.642213 (+ 12059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.663983 (+ 21770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.664174 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.645473 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:40.646598 19041 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:40.647993 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:40.651168 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 412225us
I20260709 12:17:40.651607 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:40.656296 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:40.657225 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.658118 18372 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.633857 21193 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:40.638388 21269 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.616546 21210 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:40.611505 20956 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:40.626140 20509 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.665634 18295 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:40.950212 18367 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:17:40.732277 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.875586 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 218038us
I20260709 12:17:40.952297 20924 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.954550 20914 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.954550 21209 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:40.955492 21273 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:40.956218 21209 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:40.956316 21255 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.956630 21209 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:40.988144 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:40.829526 18263 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.910s (SENT)
W20260709 12:17:40.992708 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 163224us
I20260709 12:17:40.876066 21078 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.119s
I20260709 12:17:40.796669 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:40.997118 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2122) took 1365 ms. Trace:
I20260709 12:17:40.997567 18444 rpcz_store.cc:276] 0709 12:17:39.632080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.657589 (+1025509us) service_pool.cc:224] Handling call
0709 12:17:40.997038 (+339449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.999398 18444 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:41.031096 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1682) took 1635 ms. Trace:
I20260709 12:17:41.031687 18444 rpcz_store.cc:276] 0709 12:17:39.395385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.998611 (+1603226us) service_pool.cc:224] Handling call
0709 12:17:40.999001 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.030745 (+ 31744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.031023 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.789291 19039 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:41.033619 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:41.034232 19039 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:41.035151 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:41.036325 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.989019 18367 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:41.048354 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:40.644162 20946 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:40.719475 21163 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.293s
I20260709 12:17:41.014418 20956 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.718511 21219 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:40.788331 21270 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.116084 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:41.117002 18445 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:41.117831 18298 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:17:40.717521 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52171.6us
I20260709 12:17:41.016794 21269 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.916796 21277 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.143s
W20260709 12:17:41.019351 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.018816 21193 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:40.883826 21185 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.882519 18440 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:17:40.890901 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:41.141778 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.142261 18448 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.001753 20509 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.058472 21209 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.051398 21212 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:17:40.956897 21255 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.066190 21191 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:40.958741 21114 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 12:17:41.149343 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.070104 21239 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:40.928546 21264 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.326s	user 0.000s	sys 0.000s
I20260709 12:17:40.643003 21195 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:40.929301 21265 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 12:17:41.073576 21236 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:40.875277 21168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.012s	sys 0.019s
I20260709 12:17:41.172683 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:40.788089 21164 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.111827 18368 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:40.665243 18194 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:40.948597 21222 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.114137 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:40.642939 21259 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.119707 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.122159 18297 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:41.037137 19039 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:17:40.884291 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 791) took 2697 ms. Trace:
I20260709 12:17:40.891058 20998 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:41.054234 20924 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.145509 18338 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.646s (SENT) [suppressed 27 similar messages]
I20260709 12:17:41.146171 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1692) took 1748 ms. Trace:
W20260709 12:17:41.179207 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59603.1us
I20260709 12:17:41.179741 18445 rpcz_store.cc:276] 0709 12:17:39.397778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.665947 (+1268169us) service_pool.cc:224] Handling call
0709 12:17:41.146102 (+480155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.180147 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.073019 21053 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.114638 21078 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.159443 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.196331 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:41.196589 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.177505 18194 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:41.160861 18444 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:17:41.164706 18371 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.760s	user 0.006s	sys 0.000s
I20260709 12:17:41.055333 21260 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:41.166002 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 802) took 1916 ms. Trace:
W20260709 12:17:41.166445 18300 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.005s
I20260709 12:17:41.172228 18448 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
W20260709 12:17:41.199651 18300 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.011s	sys 0.005s
I20260709 12:17:41.199824 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1688) took 1803 ms. Trace:
I20260709 12:17:41.150839 21185 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.182067 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:41.184176 19037 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:41.198608 18371 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:40.660804 18374 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0931a54c049145dfad672173c0b29a40"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:41.217223 18374 raft_consensus.cc:606] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:41.217649 18374 raft_consensus.cc:614] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:40.650149 21159 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.201083 19039 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:41.200390 18448 rpcz_store.cc:276] 0709 12:17:39.396827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.665611 (+1268784us) service_pool.cc:224] Handling call
0709 12:17:41.199766 (+534155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.172542 21114 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:17:41.227315 18339 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.004s	sys 0.023s
I20260709 12:17:41.210878 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:41.219918 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1686) took 1823 ms. Trace:
I20260709 12:17:41.051111 21258 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.143s
I20260709 12:17:41.228220 18448 rpcz_store.cc:276] 0709 12:17:39.396365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.218892 (+1822527us) service_pool.cc:224] Handling call
0709 12:17:41.219287 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.219529 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.219564 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:41.219619 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:41.219663 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.219734 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.219855 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.231374 18448 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:41.232180 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1687) took 1835 ms. Trace:
W20260709 12:17:41.227685 18339 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.004s	sys 0.023s
I20260709 12:17:41.227547 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1683) took 1831 ms. Trace:
I20260709 12:17:41.233352 18444 rpcz_store.cc:276] 0709 12:17:39.395640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.146551 (+1750911us) service_pool.cc:224] Handling call
0709 12:17:41.160587 (+ 14036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.198147 (+ 37560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.227494 (+ 29347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.199508 19040 rpcz_store.cc:276] 0709 12:17:39.249982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.317401 (+ 67419us) service_pool.cc:224] Handling call
0709 12:17:39.320174 (+  2773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:39.326995 (+  6821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:39.327029 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:39.327551 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:17:39.756788 (+429237us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:39.756845 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.756929 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.165804 (+1408875us) raft_consensus.cc:1685] finished
0709 12:17:41.165828 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.165944 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":714,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":412548,"threads_started":1,"wal-append.queue_time_us":1202241}
W20260709 12:17:41.260855 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.237643 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:41.232664 18448 rpcz_store.cc:276] 0709 12:17:39.396587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.230568 (+1833981us) service_pool.cc:224] Handling call
0709 12:17:41.230947 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.231866 (+   919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.232111 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.237146 18194 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 4 to 5, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:41.144879 21270 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:41.232993 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52833.2us
I20260709 12:17:41.278048 18293 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.209825 21260 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.194509 21275 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.277367 19041 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:41.281912 18194 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:41.226738 21163 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:41.284493 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 88003.9us
I20260709 12:17:41.210351 21053 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.316740 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.320276 18448 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:41.321367 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1691) took 1923 ms. Trace:
I20260709 12:17:41.321873 18448 rpcz_store.cc:276] 0709 12:17:39.397545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.283566 (+1886021us) service_pool.cc:224] Handling call
0709 12:17:41.283853 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.321011 (+ 37158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.321295 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.315902 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:41.275914 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 642483us
W20260709 12:17:41.324234 18339 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.556s (SENT)
I20260709 12:17:41.199712 21236 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:17:41.327418 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1689) took 1930 ms. Trace:
I20260709 12:17:41.328346 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2105) took 2143 ms. Trace:
I20260709 12:17:41.338873 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:41.320039 21053 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.198506 20924 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.195566 20998 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.341162 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:41.277737 21212 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.318885 21275 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.336409 18194 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.363181 18444 rpcz_store.cc:276] 0709 12:17:39.397047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.276955 (+1879908us) service_pool.cc:224] Handling call
0709 12:17:41.326760 (+ 49805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.326985 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.327022 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:41.327080 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:41.327131 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.327200 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.327347 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.177392 21164 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.323390 21260 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:41.225317 21078 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.227286 21114 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:41.235564 21277 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.380069 21278 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.392180 21269 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.392827 21277 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.393321 21277 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.377306 18293 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:41.364070 18447 rpcz_store.cc:276] 0709 12:17:39.184596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.294554 (+109958us) service_pool.cc:224] Handling call
0709 12:17:39.294947 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:39.295367 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:39.295408 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:39.295955 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:17:40.253347 (+957392us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:40.253414 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.253533 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.328101 (+1074568us) raft_consensus.cc:1685] finished
0709 12:17:41.328130 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.328277 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14319,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4349,"thread_start_us":924460,"threads_started":1,"wal-append.queue_time_us":1184980}
I20260709 12:17:41.318751 21163 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:41.379168 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54995.9us
I20260709 12:17:41.387995 18194 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:41.372023 21053 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.370291 21212 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:41.412736 18194 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 4 to 5, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 5 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:17:41.414925 18339 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.724s (SENT)
I20260709 12:17:41.415357 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:41.415547 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:41.413740 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.409565 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:41.380916 21276 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.374145 21275 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.391448 21293 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 12:17:41.491636 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76751.6us
W20260709 12:17:41.503712 18339 leader_election.cc:336] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.201s (SENT)
I20260709 12:17:41.504238 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:41.490432 18194 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:41.399696 21163 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:41.506170 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190355us
I20260709 12:17:41.404127 21278 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.540163 18194 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:41.417582 21212 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:41.558329 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 148828us
I20260709 12:17:41.375988 21287 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 12:17:41.566098 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:41.505602 21276 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.500334 21273 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:41.508078 21219 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:41.569638 18194 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 5 to 6, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 6 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.490381 21236 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.507321 21255 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:41.600535 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 96839.8us
I20260709 12:17:41.516368 21288 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:41.619925 18194 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 5 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 7 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.572811 21274 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.576701 21276 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:41.626096 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.629308 18194 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:41.630677 18371 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:41.571882 21210 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:41.628662 18339 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.233s (SENT)
I20260709 12:17:41.623549 21288 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.556414 21278 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:41.630208 21272 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.257s	user 0.003s	sys 0.010s
I20260709 12:17:41.623572 21255 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.637573 18194 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.640345 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:41.622915 21195 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:41.631158 21273 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.632330 21159 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.643224 18371 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.625344 21296 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.649777 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:41.651804 21283 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:17:41.651898 21164 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.624655 21044 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:17:41.652443 21273 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:41.652985 21255 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.663192 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:41.635492 21185 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.653808 21183 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.578275 21114 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.637070 21295 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.672847 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1621) took 1035 ms. Trace:
W20260709 12:17:41.673734 21282 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:17:41.632853 21274 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.641067 21288 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:41.689291 18339 leader_election.cc:336] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.516s (SENT)
I20260709 12:17:41.679163 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.621579 21236 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.691391 18371 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:41.680809 18370 rpcz_store.cc:276] 0709 12:17:40.637477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.637581 (+   104us) service_pool.cc:224] Handling call
0709 12:17:40.648784 (+ 11203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:40.670647 (+ 21863us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:17:40.670692 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:41.049980 (+379288us) log.cc:844] Serialized 26 byte log entry
0709 12:17:41.161888 (+111908us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:41.161941 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.162025 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.672637 (+510612us) raft_consensus.cc:1685] finished
0709 12:17:41.672658 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.672787 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31800,"raft.queue_time_us":68569,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":101373,"thread_start_us":111394,"threads_started":1,"wal-append.queue_time_us":186708}
I20260709 12:17:41.690336 21273 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.692601 21273 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:41.692524 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:41.693127 21273 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:41.694562 18442 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.000s
I20260709 12:17:41.639587 21293 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:41.655802 21296 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.695070 18442 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:41.700062 21183 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.703583 20998 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.703928 21270 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.716483 21270 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.705502 21255 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.706563 18194 catalog_manager.cc:5697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 3, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:41.713361 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 374581us
W20260709 12:17:41.714255 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2098) took 2528 ms (client timeout 1780 ms). Trace:
I20260709 12:17:41.715585 21297 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.657804 21210 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:17:41.729809 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:41.680605 21295 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:41.728068 18442 rpcz_store.cc:269] 0709 12:17:39.180893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.181004 (+   111us) service_pool.cc:224] Handling call
0709 12:17:41.113958 (+1932954us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:41.708771 (+594813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2560768}
I20260709 12:17:41.733860 18194 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.747192 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.748163 18369 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:41.749677 18369 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:17:41.752466 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 173987us
I20260709 12:17:41.708194 21299 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.752861 18194 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 3 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.753866 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.755627 18369 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:41.678565 21298 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:17:41.714102 21296 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.756522 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.754431 18412 leader_election.cc:400] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:41.757354 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.757573 18412 leader_election.cc:403] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:41.758230 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.760291 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.726230 21114 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:41.763055 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:41.763715 19040 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:41.732725 21078 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.764643 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:41.726130 21292 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:41.767280 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:41.177320 19036 rpcz_store.cc:276] 0709 12:17:38.092595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.103709 (+ 11114us) service_pool.cc:224] Handling call
0709 12:17:38.104112 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:38.104535 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.104567 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:38.355025 (+250458us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b18002671c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:39.550824 (+1195799us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b2c000f0768. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:39.551471 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 12:17:40.554014 (+1002543us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:40.603366 (+ 49352us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.640757 (+ 37391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.645160 (+  4403us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:40.647402 (+  2242us) raft_consensus.cc:1685] finished
0709 12:17:40.656741 (+  9339us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.719762 (+ 63021us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22842,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":29761,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":2664,"spinlock_wait_cycles":12653184,"thread_start_us":1002124,"threads_started":1,"wal-append.queue_time_us":672960}
I20260709 12:17:41.768239 18194 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:41.767267 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.752398 20998 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:41.752264 20973 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:41.771706 18371 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:41.751856 21295 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.774040 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.775106 21278 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.779574 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:41.779775 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1614) took 1144 ms. Trace:
I20260709 12:17:41.780347 20956 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:17:41.767534 21239 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:41.782294 19040 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:41.783392 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.784282 21290 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:17:41.756417 21299 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:41.798698 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.817204 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.762027 21049 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:41.803136 18194 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 3, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:41.805500 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:41.805074 18366 rpcz_store.cc:276] 0709 12:17:40.635475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.635674 (+   199us) service_pool.cc:224] Handling call
0709 12:17:40.635974 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:40.644515 (+  8541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.644548 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:40.657477 (+ 12929us) log.cc:844] Serialized 26 byte log entry
0709 12:17:41.228140 (+570663us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:41.228183 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.228248 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.779572 (+551324us) raft_consensus.cc:1685] finished
0709 12:17:41.779590 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.779722 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3939,"raft.queue_time_us":30125,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":205189,"thread_start_us":309559,"threads_started":1,"wal-append.queue_time_us":526086}
I20260709 12:17:41.812444 18445 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.816408 21299 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.819962 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.817922 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 3.
I20260709 12:17:41.812104 21271 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.826396 18372 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:41.827035 18194 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 4, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:41.830452 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:41.787914 21185 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:17:41.834218 21193 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:41.841579 18412 connection.cc:256] RPC call timeout handler was delayed by -1.3138878510001177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
I20260709 12:17:41.842931 18194 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 2 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:41.823387 21270 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.854123 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:41.855540 21308 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.856200 21308 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:41.858278 21304 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:17:41.854950 18372 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:41.822788 20973 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:41.861353 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57787.3us
I20260709 12:17:41.813936 20998 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.867851 18366 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:41.841480 21239 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:41.843652 21193 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.860235 21270 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:41.875823 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0303878510001141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:17:41.876133 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0303878510001141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:17:41.876348 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0132878510000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
W20260709 12:17:41.878187 21303 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:17:41.848381 21288 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.879343 21026 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.876811 21195 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.880067 21267 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.524s	user 0.020s	sys 0.035s
I20260709 12:17:41.880124 21288 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.880842 21274 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:41.859103 21308 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.847146 21219 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:41.866437 21183 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.832135 21275 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:41.889401 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:41.861449 21271 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.885017 21204 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:41.891503 18372 raft_consensus.cc:2376] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6. Re-sending same reply.
I20260709 12:17:41.883455 21195 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.870965 21164 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:41.877194 21270 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.876997 21193 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:41.996155 18371 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 4. Current term is 5. Ops: [3.2-4.3]
W20260709 12:17:42.257894 18339 leader_election.cc:336] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.595s (SENT)
W20260709 12:17:42.134033 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:42.252175 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.833176 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:41.999017 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 122252us
I20260709 12:17:42.834643 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:42.589727 21295 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.000s
W20260709 12:17:42.129806 21309 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 12:17:42.350317 21277 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.000s
W20260709 12:17:42.274240 21053 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.856s	user 0.007s	sys 0.003s
W20260709 12:17:42.546007 21212 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.928s	user 0.008s	sys 0.003s
W20260709 12:17:42.574735 21163 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.036s	user 0.009s	sys 0.002s
W20260709 12:17:42.296146 21260 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.000s
I20260709 12:17:42.290081 21275 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:41.918066 21183 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:42.886828 18369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.007s	sys 0.007s
W20260709 12:17:42.966043 18369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.007s	sys 0.007s
W20260709 12:17:42.876156 21270 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.812s	user 0.010s	sys 0.000s
W20260709 12:17:42.570930 21276 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.934s	user 0.008s	sys 0.001s
I20260709 12:17:42.811345 21204 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:42.874437 21114 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.109s	user 0.009s	sys 0.002s
I20260709 12:17:42.269302 21191 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:42.448331 20998 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:42.360954 21271 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:42.834745 21193 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:42.960043 19041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.013s	sys 0.036s
W20260709 12:17:43.038978 19041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.251s	user 0.013s	sys 0.036s
I20260709 12:17:42.309945 21219 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:42.061533 21164 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:42.585704 21296 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.798s	user 0.007s	sys 0.003s
W20260709 12:17:42.886555 21305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.007s	sys 0.005s
I20260709 12:17:41.992337 21273 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:42.882367 19036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.008s	sys 0.039s
I20260709 12:17:42.078444 20973 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:42.884078 21053 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:43.050820 18449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2115) took 3801 ms (client timeout 2020 ms). Trace:
I20260709 12:17:42.887135 21277 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:42.962795 21260 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.048926 19038 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:42.959872 21212 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:42.962953 21163 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.057468 18449 rpcz_store.cc:269] 0709 12:17:39.273026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.281540 (+  8514us) service_pool.cc:224] Handling call
0709 12:17:43.050657 (+3769117us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400354dd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:43.050754 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512328448}
I20260709 12:17:43.041965 21271 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.059024 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 727) took 3758 ms. Trace:
I20260709 12:17:43.039559 21195 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:43.049955 19036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.008s	sys 0.040s
W20260709 12:17:43.045113 21305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.008s	sys 0.006s
I20260709 12:17:43.063220 19038 rpcz_store.cc:276] 0709 12:17:39.300745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.048213 (+3747468us) service_pool.cc:224] Handling call
0709 12:17:43.048574 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.058755 (+ 10181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.058967 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:43.060839 18264 connection.cc:256] RPC call timeout handler was delayed by -2.2795212609999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:17:42.886883 21295 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.036160 21191 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.043303 21193 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.032585 21270 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.067157 18264 connection.cc:256] RPC call timeout handler was delayed by -2.2291212609998183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 12:17:43.032464 21276 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.069384 18264 connection.cc:256] RPC call timeout handler was delayed by -1.9366212609998001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:17:43.039027 21204 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:43.071796 18264 connection.cc:256] RPC call timeout handler was delayed by -1.6864212609998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:17:43.067332 21299 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.232s	user 0.012s	sys 0.000s
I20260709 12:17:43.045341 21296 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.082522 18264 connection.cc:256] RPC call timeout handler was delayed by -1.1347212609998678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2804s for a response
W20260709 12:17:43.084095 18264 connection.cc:256] RPC call timeout handler was delayed by -1.0987212609998096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2844s for a response
W20260709 12:17:43.084424 18264 connection.cc:256] RPC call timeout handler was delayed by -1.0438212609999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2905s for a response
I20260709 12:17:43.087946 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 807) took 1333 ms. Trace:
I20260709 12:17:43.090106 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.090649 18298 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:43.091477 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.092317 18295 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:43.088497 19041 rpcz_store.cc:276] 0709 12:17:41.754678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.754794 (+   116us) service_pool.cc:224] Handling call
0709 12:17:41.765454 (+ 10660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:41.769227 (+  3773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.769271 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:41.787656 (+ 18385us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.039471 (+1251815us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:43.039520 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.039588 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.087663 (+ 48075us) raft_consensus.cc:1685] finished
0709 12:17:43.087693 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.087869 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3346,"raft.queue_time_us":15449,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":1251476,"threads_started":1,"wal-append.queue_time_us":1262099}
I20260709 12:17:43.092629 21204 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:43.098593 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.099732 18294 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:43.096787 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1755) took 1292 ms. Trace:
I20260709 12:17:43.083386 21114 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:43.104208 21114 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.083825 21299 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.107157 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 810) took 1349 ms. Trace:
I20260709 12:17:43.107601 21191 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.108057 21114 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.109048 21212 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.100456 18368 rpcz_store.cc:276] 0709 12:17:41.804560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.806005 (+  1445us) service_pool.cc:224] Handling call
0709 12:17:41.821141 (+ 15136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:41.821563 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:41.834965 (+ 13402us) log.cc:844] Serialized 14 byte log entry
0709 12:17:42.254524 (+419559us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.272356 (+ 17832us) log.cc:844] Serialized 26 byte log entry
0709 12:17:42.272533 (+   177us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:42.272581 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.272666 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.096498 (+823832us) raft_consensus.cc:1685] finished
0709 12:17:43.096540 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.096710 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6568,"prepare.queue_time_us":13290,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":140647,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":20434,"thread_start_us":276783,"threads_started":1,"wal-append.queue_time_us":1097743}
W20260709 12:17:43.111249 21316 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:17:43.112028 21164 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.112627 21276 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.113270 21270 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.113921 21260 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.075852 21271 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 12:17:43.114432 21284 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.744s	user 0.051s	sys 0.076s
I20260709 12:17:43.115044 21273 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.115154 21053 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.116791 21277 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.119711 19036 rpcz_store.cc:276] 0709 12:17:41.758196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.819185 (+ 60989us) service_pool.cc:224] Handling call
0709 12:17:41.848598 (+ 29413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:41.874231 (+ 25633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.877294 (+  3063us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.093586 (+216292us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.074774 (+981188us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:43.075562 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.083334 (+  7772us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.105580 (+ 22246us) raft_consensus.cc:1685] finished
0709 12:17:43.106861 (+  1281us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.107071 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":17390,"thread_start_us":764093,"threads_started":1,"wal-append.queue_time_us":791631}
I20260709 12:17:43.131019 21209 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:43.131327 21163 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.131593 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1751) took 1365 ms. Trace:
I20260709 12:17:43.132100 18369 rpcz_store.cc:276] 0709 12:17:41.765754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.765982 (+   228us) service_pool.cc:224] Handling call
0709 12:17:41.766358 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:41.826655 (+ 60297us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:41.826689 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:41.827122 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:17:42.966522 (+1139400us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:42.966573 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.966659 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.131323 (+164664us) raft_consensus.cc:1685] finished
0709 12:17:43.131349 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.131522 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5227,"raft.queue_time_us":69641,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":15993,"thread_start_us":1179669,"threads_started":2,"wal-append.queue_time_us":1126476}
I20260709 12:17:43.112637 21183 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.122509 21191 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.125869 18374 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.134634 21191 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.135129 21191 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:43.136672 21299 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.137152 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 539) took 1281 ms. Trace:
I20260709 12:17:43.137566 18374 rpcz_store.cc:276] 0709 12:17:41.855803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.880979 (+ 25176us) service_pool.cc:224] Handling call
0709 12:17:41.881295 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.134469 (+1253174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.136912 (+  2443us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000c85d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.137094 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861856512}
I20260709 12:17:43.136142 21191 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.124593 21295 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.124506 21296 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:43.149389 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 891443us
W20260709 12:17:43.150262 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.622s (SENT)
I20260709 12:17:43.145433 21191 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.158427 21065 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.158865 21065 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.159816 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 801) took 3910 ms. Trace:
I20260709 12:17:43.159775 21065 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.160266 19037 rpcz_store.cc:276] 0709 12:17:39.249718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.279763 (+ 30045us) service_pool.cc:224] Handling call
0709 12:17:39.301176 (+ 21413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.216663 (+1915487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.159576 (+1942913us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400264654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.159760 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4329894272}
I20260709 12:17:43.160916 21065 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:43.168182 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 729) took 3825 ms (client timeout 1910 ms). Trace:
W20260709 12:17:43.168625 19042 rpcz_store.cc:269] 0709 12:17:39.343063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.167931 (+3824868us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:43.168080 (+   149us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:17:43.173400 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1681) took 3778 ms (client timeout 2844 ms). Trace:
W20260709 12:17:43.173839 18440 rpcz_store.cc:269] 0709 12:17:39.395028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.401694 (+  6666us) service_pool.cc:224] Handling call
0709 12:17:43.173226 (+3771532us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003b15d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:43.173333 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2703209856}
I20260709 12:17:43.174957 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.175841 18374 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:43.175923 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.175866 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.179952 18369 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:43.180662 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.181651 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:43.182132 19006 connection.cc:256] RPC call timeout handler was delayed by -1.9737911959998655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2493s for a response
I20260709 12:17:43.182263 18368 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 3.
W20260709 12:17:43.182673 19006 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.556s (SENT)
I20260709 12:17:43.183372 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:43.183990 18374 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.184038 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.184839 18372 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:43.157820 21322 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.186084 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.186434 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.186753 18368 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:43.188617 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.189028 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.189599 19035 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:43.189509 21322 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.190157 19035 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 806) took 3806 ms (client timeout 4704 ms). Trace:
W20260709 12:17:43.190637 21289 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.801s	user 0.002s	sys 0.013s
W20260709 12:17:43.190585 19035 rpcz_store.cc:269] 0709 12:17:39.383597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.140591 (+3756994us) service_pool.cc:224] Handling call
0709 12:17:43.190103 (+ 49512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:43.192668 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.185889 20946 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:43.196738 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 362188us
I20260709 12:17:43.198150 20946 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.204901 18374 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:43.198712 21326 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.199934 21230 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.226805 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:43.227579 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6.
I20260709 12:17:43.228991 18374 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:43.214805 20946 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.229298 21325 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:17:43.227310 21326 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.239603 21296 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.239866 21326 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.239980 21296 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.240913 21296 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.242046 21296 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.248970 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.251351 19035 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.251793 21183 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:43.251395 21212 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:17:43.252537 18374 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:17:43.253403 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:43.254470 18291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2070) took 2602 ms (client timeout 1998 ms). Trace:
I20260709 12:17:43.254930 21160 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:43.255575 21160 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:43.256235 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.257388 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.257963 21160 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:43.258721 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:43.254869 18291 rpcz_store.cc:269] 0709 12:17:40.651964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.254263 (+2602299us) service_pool.cc:211] Skipping call since client already timed out
0709 12:17:43.254369 (+   106us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:17:43.252465 21183 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:43.257992 19035 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:43.260151 21183 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:43.260577 21183 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.261369 21183 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.262480 21183 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:43.254895 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.267755 20924 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:43.268090 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.271157 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.286082 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.286303 21326 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.286916 18374 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:43.288108 21237 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:43.288636 21237 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:43.289710 21237 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.290753 21237 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.291661 20924 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.292155 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.293035 21325 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:43.293401 21325 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:43.293732 21325 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.296054 21278 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:43.296454 21278 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:43.297248 19044 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:43.297304 21278 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.298447 21278 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:43.299285 18263 connection.cc:256] RPC call timeout handler was delayed by -1.4491084129999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
I20260709 12:17:43.299897 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.299970 21236 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:43.300144 18263 connection.cc:256] RPC call timeout handler was delayed by -1.2396112950000315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:17:43.301044 21160 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:43.301793 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.301618 21160 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.302472 21295 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.302604 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1685) took 3906 ms. Trace:
I20260709 12:17:43.302950 21295 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.251345 20946 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.303751 21160 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.304286 21160 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.304885 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 469) took 3688 ms. Trace:
I20260709 12:17:43.305112 21295 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.305572 20946 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.297914 21255 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.307009 21255 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.308225 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.307979 21325 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.310000 21325 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.316272 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.318161 21260 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.318549 21260 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.318845 21260 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.322118 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.322790 19040 raft_consensus.cc:2368] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5.
I20260709 12:17:43.322917 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.323772 19035 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:43.326074 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.327035 19035 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:43.305388 18300 rpcz_store.cc:276] 0709 12:17:39.616483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.250106 (+633623us) service_pool.cc:224] Handling call
0709 12:17:40.277353 (+ 27247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.617658 (+340305us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:17:41.200175 (+582517us) log.cc:844] Serialized 14 byte log entry
0709 12:17:41.572678 (+372503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:41.572730 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:17:41.572782 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.190109 (+1617327us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:43.304627 (+114518us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.304670 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.304792 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27649,"prepare.queue_time_us":666830,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2629,"raft.queue_time_us":236196,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":43990,"spinlock_wait_cycles":816828544,"thread_start_us":926590,"threads_started":3,"wal-append.queue_time_us":180944}
I20260709 12:17:43.306267 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1615) took 2670 ms. Trace:
I20260709 12:17:43.302985 18445 rpcz_store.cc:276] 0709 12:17:39.396142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.180658 (+1784516us) service_pool.cc:224] Handling call
0709 12:17:41.181031 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.828479 (+647448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.302364 (+1473885us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54001a5c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.302525 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448231808}
W20260709 12:17:43.238471 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53141.9us
I20260709 12:17:43.345009 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:43.347185 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.347296 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47497us
W20260709 12:17:43.347834 18339 leader_election.cc:336] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.628s (SENT)
I20260709 12:17:43.348213 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:43.348213 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.349848 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.350065 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.350644 18445 raft_consensus.cc:2376] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5. Re-sending same reply.
W20260709 12:17:43.351272 18339 leader_election.cc:336] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.646s (SENT)
W20260709 12:17:43.351890 18339 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.867s (SENT)
I20260709 12:17:43.352313 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.352957 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.353659 18445 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:43.259122 20973 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:43.356832 19037 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:43.357589 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:43.358145 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.360977 18441 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:43.359788 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.362007 18445 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:43.363438 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.365121 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.366623 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:43.367376 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 162051us
I20260709 12:17:43.368309 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.369870 18447 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:43.331411 18373 rpcz_store.cc:276] 0709 12:17:40.635791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.635904 (+   113us) service_pool.cc:224] Handling call
0709 12:17:40.636270 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:41.147479 (+511209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.147534 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:41.171834 (+ 24300us) log.cc:844] Serialized 52 byte log entry
0709 12:17:41.407765 (+235931us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:41.506678 (+ 98913us) log.cc:844] Serialized 14 byte log entry
0709 12:17:41.627487 (+120809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.305890 (+1678403us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54001676d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.305933 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.305976 (+    43us) raft_consensus.cc:1685] finished
0709 12:17:43.306066 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.306190 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41105,"prepare.queue_time_us":98773,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":155220,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":15870,"spinlock_wait_cycles":1029238528,"thread_start_us":287657,"threads_started":2,"wal-append.queue_time_us":452386,"wal-append.run_cpu_time_us":3432,"wal-append.run_wall_time_us":1243172}
W20260709 12:17:43.373406 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 184724us
I20260709 12:17:43.364205 18445 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:43.376118 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.377410 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.378074 18448 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:43.379868 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.380669 18445 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:17:43.384320 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.385051 18447 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 4: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:43.388172 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.388690 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.388691 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.388742 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.390793 18443 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:43.391920 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.392398 21333 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:43.392971 21333 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.394100 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.396072 18338 leader_election.cc:400] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5.
I20260709 12:17:43.396440 18338 leader_election.cc:403] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:43.397549 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.403056 18448 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:43.404289 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.405580 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.407035 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.408917 18440 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:43.388993 18445 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:43.407671 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52244.6us
I20260709 12:17:43.410866 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.346500 20914 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.412853 18445 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.416981 18367 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:17:43.338469 21259 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:43.371654 21191 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.356258 21168 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:43.417022 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.431704 18447 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:43.392515 18443 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.374455 21325 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:43.355984 21210 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:43.432700 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.424069 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.437193 18447 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.437599 18440 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:43.426307 18263 leader_election.cc:336] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.571s (SENT)
I20260709 12:17:43.383765 20998 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:43.440979 20998 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.441890 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.442451 18446 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.382068 21212 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:43.443625 21212 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.444780 21333 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:43.444525 21212 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.383301 21295 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.411139 21323 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.407919 21271 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.447299 21323 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.447753 21323 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.446784 21295 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.411139 21314 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:43.440225 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.464469 18441 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:43.438416 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.455653 18446 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:43.465953 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:43.466323 18263 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.771s (SENT)
I20260709 12:17:43.466512 18446 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:43.467303 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.468039 18446 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:17:43.474311 18263 leader_election.cc:336] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 2.049s (SENT)
I20260709 12:17:43.474803 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:43.477056 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.482158 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.483685 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.457122 18448 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:43.414934 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.488955 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.389955 20973 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.486364 18263 leader_election.cc:336] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 2.071s (SENT)
W20260709 12:17:43.446830 21328 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:17:43.435871 21273 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:43.541647 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.542512 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2069) took 2891 ms. Trace:
I20260709 12:17:43.542762 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:43.443071 20998 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.495316 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:43.545694 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50481.7us
I20260709 12:17:43.546582 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.485596 18447 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:43.546433 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190463us
I20260709 12:17:43.543004 18296 rpcz_store.cc:276] 0709 12:17:40.651612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.148273 (+2496661us) service_pool.cc:224] Handling call
0709 12:17:43.151478 (+  3205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.151754 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:43.369565 (+217811us) log.cc:844] Serialized 14 byte log entry
0709 12:17:43.542156 (+172591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:43.542209 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:43.542253 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.542317 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.542458 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":239729,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":383752,"threads_started":2}
I20260709 12:17:43.459769 21295 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.547799 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.548458 18447 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:43.496382 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.549145 18446 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:43.496789 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.429950 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.498917 18440 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:17:43.499812 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.408917 21160 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:17:43.501572 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.503789 18443 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:43.557309 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 70989.8us
I20260709 12:17:43.557574 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.558655 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.485139 21183 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.561149 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.512495 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:43.516286 19044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 554) took 4346 ms (client timeout 1828 ms). Trace:
I20260709 12:17:43.517656 18294 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:43.564174 18294 pending_rounds.cc:85] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 2
I20260709 12:17:43.564502 18294 pending_rounds.cc:107] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:43.564803 18294 raft_consensus.cc:2889] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:43.518781 18295 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:43.566110 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.567386 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.519534 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.520344 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.550328 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:43.522796 18293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2063) took 2889 ms (client timeout 3319 ms). Trace:
I20260709 12:17:43.523633 18369 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:43.524920 21299 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:43.525301 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:43.526726 18367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1624) took 2385 ms (client timeout 1969 ms). Trace:
W20260709 12:17:43.573395 18293 rpcz_store.cc:269] 0709 12:17:40.633407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.633541 (+   134us) service_pool.cc:224] Handling call
0709 12:17:43.522583 (+2889042us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540040d654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:43.522701 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451637504}
I20260709 12:17:43.528093 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.585409 19042 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.538664 21294 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:43.539141 21255 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.539899 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.424124 21322 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:43.549980 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1020855729998402s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3027s for a response
I20260709 12:17:43.596566 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.552408 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.559697 21222 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:43.485189 20914 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.556594 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.620967 18447 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:43.506026 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.559942 18443 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:43.623394 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.511971 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:43.521972 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1754) took 1719 ms. Trace:
I20260709 12:17:43.624478 18371 rpcz_store.cc:276] 0709 12:17:41.802501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.802774 (+   273us) service_pool.cc:224] Handling call
0709 12:17:41.818812 (+ 16038us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:42.832504 (+1013692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.521647 (+689143us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54003f2354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:43.521902 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407250176}
W20260709 12:17:43.562184 19044 rpcz_store.cc:269] 0709 12:17:39.170204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.170327 (+   123us) service_pool.cc:224] Handling call
0709 12:17:43.516020 (+4345693us) spinlock_profiling.cc:243] Waited 4.11 s on lock 0x7b54004777d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:43.516162 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9456482048}
I20260709 12:17:43.502157 21163 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.573535 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.625901 18300 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:43.561882 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.626223 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.626413 18443 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.541771 21219 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:43.641403 18446 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:43.574651 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:43.579306 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.580988 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.644469 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:43.458803 21210 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.645478 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:43.646204 18298 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:43.646651 21210 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:43.586874 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.647464 18298 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:43.594341 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.597584 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.470403 21314 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.599779 20924 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.648912 21314 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.650148 20924 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:43.650651 20924 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.561527 21305 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:43.618422 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:43.648687 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51171.7us
I20260709 12:17:43.574406 18299 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:43.652869 18299 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:43.572443 21053 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.654779 18298 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:43.656013 18298 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.578593 21183 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.661437 21183 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.631956 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.662307 18441 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:43.663127 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.632740 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.643038 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.665092 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.665151 21314 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.665686 18447 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:43.666493 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.644624 18440 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:43.668669 18440 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:43.668921 21314 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.669747 18440 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.670061 18446 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
W20260709 12:17:43.584543 18367 rpcz_store.cc:269] 0709 12:17:41.141239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.141415 (+   176us) service_pool.cc:224] Handling call
0709 12:17:43.526478 (+2385063us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54001f9fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:43.526628 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5229722880}
I20260709 12:17:43.647403 18295 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
I20260709 12:17:43.486639 20946 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:17:43.671571 18446 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:43.672557 18440 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:43.673962 18440 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.651087 20924 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:43.651724 18292 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
W20260709 12:17:43.626879 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65066.4us
I20260709 12:17:43.680086 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.681108 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.682646 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.648679 18293 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:43.674510 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.675026 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.685123 21325 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.686457 21325 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.686920 21325 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.685648 21183 raft_consensus.cc:2764] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:43.685712 21292 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:43.691012 21292 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.691519 21292 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.692806 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.694767 21183 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:43.695247 21183 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.696168 21183 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.698266 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.574452 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.699168 18440 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:43.704308 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 363843us
I20260709 12:17:43.704952 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.705338 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:43.707185 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 132605us
I20260709 12:17:43.708267 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:43.709697 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.587142 20998 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:43.710330 18297 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:43.547152 21273 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.714012 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:43.719053 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.742782 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.705641 21227 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.592173 21294 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:43.784657 18443 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:43.620467 21222 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:43.799248 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91057.9us
W20260709 12:17:43.800089 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.631629 21255 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.704763 21044 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:43.794920 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 252214us
I20260709 12:17:43.797189 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.803829 18367 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:43.810755 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:43.811400 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.592696 21299 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.799963 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.634845 21219 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:43.578737 21260 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.808472 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.804664 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:43.811462 18367 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.820134 18366 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.797029 21330 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:43.816239 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:43.816479 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:43.741389 21053 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:43.795917 21193 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:43.698618 20924 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.813459 21296 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.003s
I20260709 12:17:43.751849 21305 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:43.827893 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.830061 18300 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:43.828266 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.828517 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.811559 21044 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:43.831838 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.806919 21222 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:43.801420 21191 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:43.832612 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.856611 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:43.812354 21255 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.822654 21219 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:43.902813 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.814291 21314 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:43.847950 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.856451 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.854417 18449 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:43.905737 19038 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:43.815596 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:43.852203 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.863868 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:43.832556 21326 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:43.906994 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54912.6us
I20260709 12:17:43.893623 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.907222 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.800630 21241 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:17:43.799579 21227 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:43.899354 21065 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.727s	user 0.000s	sys 0.009s
I20260709 12:17:43.923766 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:43.922983 20973 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:43.935642 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.936439 20924 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.936830 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.937191 21295 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.937872 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:43.939666 18299 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:43.855510 18291 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:43.940106 18372 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:43.939668 21295 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.940670 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:43.941298 18294 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:43.899855 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:43.944202 21344 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:17:43.944342 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:43.945315 18295 raft_consensus.cc:2368] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6.
I20260709 12:17:43.946274 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:43.947744 18294 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.948132 18294 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:43.948704 21260 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.897215 21193 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:43.948889 18294 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:43.949957 18294 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.950367 21193 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.979136 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:43.952306 21204 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.015683 18295 raft_consensus.cc:2368] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7.
I20260709 12:17:44.016134 19038 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:43.897087 20998 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:43.828212 21269 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.864902 21322 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:43.822165 21339 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:17:44.019824 21204 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:44.020283 21339 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.952615 21230 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:44.020735 21339 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:44.020879 21230 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.021278 21230 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:44.022080 21204 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:43.905166 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:44.022662 18296 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:44.022809 21323 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.002s
I20260709 12:17:44.023260 21323 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.840378 21168 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:43.907877 18300 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:43.908807 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.909421 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:43.903565 19039 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:43.835412 21220 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:17:43.863787 21255 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.848955 21330 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:43.920601 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:43.894219 18292 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:43.922950 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:44.021665 21339 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:43.857235 21044 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.027837 18294 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:44.033656 18442 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:44.033906 18296 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:43.905467 21314 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:44.034993 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:44.338768 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.215554 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:44.341204 18443 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:44.341435 18448 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:44.575378 19043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.012s	sys 0.014s
W20260709 12:17:44.874675 19043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.018s	sys 0.018s
I20260709 12:17:44.498728 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:44.752730 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.344893 18297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.018s
W20260709 12:17:44.876148 18297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.006s	sys 0.038s
W20260709 12:17:44.398276 18440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.016s
W20260709 12:17:44.399923 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44683.8us
I20260709 12:17:44.342283 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:44.877821 18443 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:44.785553 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.006s	sys 0.010s
I20260709 12:17:44.349663 19040 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:44.452875 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:44.879303 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1682) took 1044 ms. Trace:
W20260709 12:17:44.174222 19042 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.005s
I20260709 12:17:44.880779 19042 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:44.880360 18292 rpcz_store.cc:276] 0709 12:17:43.835278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.835469 (+   191us) service_pool.cc:224] Handling call
0709 12:17:43.893589 (+ 58120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.879032 (+985443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.879254 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.079384 19037 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.883463 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1715) took 1051 ms. Trace:
I20260709 12:17:44.883970 18445 rpcz_store.cc:276] 0709 12:17:43.831631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.908534 (+ 76903us) service_pool.cc:224] Handling call
0709 12:17:44.883402 (+974868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.885088 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:44.878535 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.006s	sys 0.010s
I20260709 12:17:44.886101 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:44.886642 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 629) took 1025 ms. Trace:
I20260709 12:17:44.886891 18445 rpcz_store.cc:276] 0709 12:17:43.860777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.885941 (+1025164us) service_pool.cc:224] Handling call
0709 12:17:44.886605 (+   664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.875569 18294 pending_rounds.cc:85] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 2
I20260709 12:17:44.887511 18294 pending_rounds.cc:107] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:44.876410 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.887755 18294 raft_consensus.cc:2889] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:44.888748 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.889209 18294 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:44.889034 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:44.889991 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:44.442878 18444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.026s	sys 0.021s
I20260709 12:17:44.195253 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:44.877049 18440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.014s	sys 0.039s
I20260709 12:17:44.891219 19038 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:44.881585 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 818) took 1533 ms (client timeout 1589 ms). Trace:
W20260709 12:17:44.891885 19042 rpcz_store.cc:269] 0709 12:17:43.348288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.348434 (+   146us) service_pool.cc:224] Handling call
0709 12:17:43.527928 (+179494us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:44.881480 (+1353552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26076544}
W20260709 12:17:44.890810 18444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.344s	user 0.028s	sys 0.043s
I20260709 12:17:44.880127 18296 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:44.893247 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:44.894120 19042 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:44.894505 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 748) took 1067 ms. Trace:
W20260709 12:17:44.112437 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 414669us
I20260709 12:17:44.894879 19042 rpcz_store.cc:276] 0709 12:17:43.827286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.893037 (+1065751us) service_pool.cc:224] Handling call
0709 12:17:44.894457 (+  1420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.895596 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:44.895798 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:44.896320 19042 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.892066 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1704) took 1346 ms. Trace:
I20260709 12:17:44.900259 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 744) took 1073 ms. Trace:
I20260709 12:17:44.900830 19040 rpcz_store.cc:276] 0709 12:17:43.826424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.034728 (+208304us) service_pool.cc:224] Handling call
0709 12:17:44.900172 (+865444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.877830 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:44.206836 21237 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.916s	user 0.009s	sys 0.002s
W20260709 12:17:44.205080 21292 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.012s
I20260709 12:17:44.901952 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:44.729851 21230 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.708s	user 0.004s	sys 0.002s
I20260709 12:17:44.912801 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:44.079312 21337 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.247s	user 0.000s	sys 0.000s
I20260709 12:17:44.929227 18445 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:44.123725 21351 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.204s	user 0.000s	sys 0.008s
W20260709 12:17:44.415470 21349 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.513s	user 0.001s	sys 0.000s
I20260709 12:17:44.933408 19042 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:44.581362 21299 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:44.842272 21325 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.155s	user 0.009s	sys 0.008s
W20260709 12:17:44.833686 21271 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.386s	user 0.010s	sys 0.000s
W20260709 12:17:44.834609 21278 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.536s	user 0.008s	sys 0.008s
I20260709 12:17:44.890631 18294 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:44.413789 21295 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.534s
I20260709 12:17:44.898816 18440 rpcz_store.cc:276] 0709 12:17:43.545506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.704324 (+158818us) service_pool.cc:224] Handling call
0709 12:17:43.743431 (+ 39107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.743934 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:43.822102 (+ 78168us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.891618 (+1069516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.891689 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:44.891755 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.891843 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.891997 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62107,"prepare.queue_time_us":84405,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":2143,"thread_start_us":1069164,"threads_started":1}
I20260709 12:17:44.412319 21314 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.930455 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 754) took 1093 ms. Trace:
I20260709 12:17:44.956714 18374 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:44.956825 19040 rpcz_store.cc:276] 0709 12:17:43.837018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.901673 (+1064655us) service_pool.cc:224] Handling call
0709 12:17:44.930393 (+ 28720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.929840 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2159) took 1015 ms. Trace:
I20260709 12:17:44.929656 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.957938 18445 rpcz_store.cc:276] 0709 12:17:43.914067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.912508 (+998441us) service_pool.cc:224] Handling call
0709 12:17:44.929758 (+ 17250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.958304 18292 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:44.781936 20973 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:44.728948 21219 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.000s
W20260709 12:17:44.568162 21358 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:17:44.934096 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 753) took 1099 ms. Trace:
I20260709 12:17:44.100447 21269 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:44.935242 21339 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.963908 19042 rpcz_store.cc:276] 0709 12:17:43.834222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.895572 (+1061350us) service_pool.cc:224] Handling call
0709 12:17:44.934010 (+ 38438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.948161 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1670) took 1434 ms. Trace:
W20260709 12:17:44.949880 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.021s	sys 0.021s
W20260709 12:17:44.966691 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.022s	sys 0.021s
I20260709 12:17:44.966581 18291 rpcz_store.cc:276] 0709 12:17:43.513869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.587050 (+ 73181us) service_pool.cc:224] Handling call
0709 12:17:44.947910 (+1360860us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b540004d654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:44.948074 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525632256}
W20260709 12:17:44.967056 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.05761e+06us
I20260709 12:17:44.968036 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:44.025213 21323 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:44.934536 21220 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:44.954758 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.973608 18294 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:17:44.974900 18296 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:44.959285 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:44.951581 21271 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5.
I20260709 12:17:44.949528 19038 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:44.926350 18300 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:44.978715 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:44.175529 21305 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:44.932863 21230 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:44.984982 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 8903d70080d24236be955433a92791c7: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:44.972301 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:44.974833 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:44.987751 19042 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 5: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:44.987761 18294 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:44.988241 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 573) took 1391 ms. Trace:
I20260709 12:17:44.988610 19042 rpcz_store.cc:276] 0709 12:17:43.597259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.972042 (+1374783us) service_pool.cc:224] Handling call
0709 12:17:44.988185 (+ 16143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.989590 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:44.990265 19042 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:44.990624 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 826) took 1322 ms. Trace:
I20260709 12:17:44.962903 21219 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:44.966161 21227 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:44.975576 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 476) took 1031 ms. Trace:
I20260709 12:17:44.890614 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.975731 18445 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:44.976640 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:44.973277 21220 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:44.990886 18294 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.990906 19042 rpcz_store.cc:276] 0709 12:17:43.668450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.989345 (+1320895us) service_pool.cc:224] Handling call
0709 12:17:44.990579 (+  1234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.020951 18298 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:45.018302 18296 rpcz_store.cc:276] 0709 12:17:43.944366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.413435 (+469069us) service_pool.cc:224] Handling call
0709 12:17:44.975520 (+562085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.022784 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 473) took 1079 ms. Trace:
I20260709 12:17:45.023085 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1716) took 1173 ms. Trace:
I20260709 12:17:44.962129 20973 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.018275 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 471) took 1075 ms. Trace:
I20260709 12:17:44.979365 18291 raft_consensus.cc:2376] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7. Re-sending same reply.
I20260709 12:17:44.970341 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:44.980695 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 755) took 1090 ms. Trace:
I20260709 12:17:44.980809 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 470) took 1038 ms. Trace:
I20260709 12:17:45.023643 18445 rpcz_store.cc:276] 0709 12:17:43.849670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.958921 (+1109251us) service_pool.cc:224] Handling call
0709 12:17:45.023028 (+ 64107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.025252 18300 rpcz_store.cc:276] 0709 12:17:43.942767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.035037 (+ 92270us) service_pool.cc:224] Handling call
0709 12:17:44.980756 (+945719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.022713 18440 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:44.949888 21325 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:44.930744 21292 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:44.983383 19037 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:45.023399 18298 rpcz_store.cc:276] 0709 12:17:43.943518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.020254 (+1076736us) service_pool.cc:224] Handling call
0709 12:17:45.020613 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.022477 (+  1864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.022712 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.024362 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2088) took 1099 ms. Trace:
W20260709 12:17:44.983959 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.970515 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:45.024667 19038 rpcz_store.cc:276] 0709 12:17:43.890422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.034766 (+144344us) service_pool.cc:224] Handling call
0709 12:17:44.980634 (+945868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.024763 18292 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:45.014441 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:44.983774 21269 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:45.028398 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 757) took 1124 ms. Trace:
I20260709 12:17:45.100796 19040 rpcz_store.cc:276] 0709 12:17:43.904238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.957679 (+1053441us) service_pool.cc:224] Handling call
0709 12:17:45.028328 (+ 70649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.039297 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1712) took 1287 ms. Trace:
I20260709 12:17:45.102301 18440 rpcz_store.cc:276] 0709 12:17:43.752206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.964263 (+1212057us) service_pool.cc:224] Handling call
0709 12:17:45.039213 (+ 74950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.039817 19039 raft_consensus.cc:2368] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8.
I20260709 12:17:45.026961 18291 rpcz_store.cc:276] 0709 12:17:43.924608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.967577 (+1042969us) service_pool.cc:224] Handling call
0709 12:17:45.024306 (+ 56729us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.029711 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 478) took 1084 ms. Trace:
I20260709 12:17:45.105216 18292 rpcz_store.cc:276] 0709 12:17:43.944843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.959346 (+1014503us) service_pool.cc:224] Handling call
0709 12:17:45.029647 (+ 70301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.066988 18294 rpcz_store.cc:276] 0709 12:17:43.943006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.989969 (+1046963us) service_pool.cc:224] Handling call
0709 12:17:44.990567 (+   598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.018024 (+ 27457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.018220 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:45.108700 21364 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:17:45.005712 21219 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:44.963913 21330 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.106211 21227 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:45.110579 21227 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:44.988626 21044 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:45.018850 21373 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:17:45.112071 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:17:45.122996 18447 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:45.116671 18300 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:45.124402 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 479) took 1179 ms. Trace:
I20260709 12:17:45.124854 18300 rpcz_store.cc:276] 0709 12:17:43.945112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.025881 (+1080769us) service_pool.cc:224] Handling call
0709 12:17:45.116460 (+ 90579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.124092 (+  7632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.124336 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.117507 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1681) took 1303 ms. Trace:
I20260709 12:17:45.126024 18293 rpcz_store.cc:276] 0709 12:17:43.814372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.821211 (+  6839us) service_pool.cc:224] Handling call
0709 12:17:43.821522 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.821767 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.821798 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.834885 (+ 13087us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.013845 (+178960us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:44.013883 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.013959 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.117224 (+1103265us) raft_consensus.cc:1685] finished
0709 12:17:45.117251 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.117425 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":119525,"threads_started":1,"wal-append.queue_time_us":1154226}
I20260709 12:17:45.090826 21295 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.026852 21367 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:45.120752 21222 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.228s	user 0.011s	sys 0.001s
I20260709 12:17:45.026266 21230 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:45.038920 20973 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.183907 18441 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:45.154400 21255 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.002s
W20260709 12:17:45.248939 21366 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:45.111508 21227 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.182018 21367 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.263154 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 813) took 1941 ms. Trace:
I20260709 12:17:45.020571 21220 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:45.287436 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 391888us
I20260709 12:17:45.290336 19043 rpcz_store.cc:276] 0709 12:17:43.322095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.333136 (+ 11041us) service_pool.cc:224] Handling call
0709 12:17:43.333511 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.333770 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:43.564333 (+230563us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.262658 (+1698325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:45.262735 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:45.262805 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.262898 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.263078 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5950,"prepare.queue_time_us":408113,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":115561,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3625,"thread_start_us":1883358,"threads_started":3,"wal-append.queue_time_us":1454627}
I20260709 12:17:45.314880 18296 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:45.321712 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 474) took 1377 ms. Trace:
I20260709 12:17:45.322990 18296 rpcz_store.cc:276] 0709 12:17:43.943829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.023102 (+1079273us) service_pool.cc:224] Handling call
0709 12:17:45.023472 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.315488 (+292016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.321627 (+  6139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.187155 21295 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.297981 21373 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:45.367934 21294 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.369084 21294 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.371914 21325 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:45.372650 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 547) took 1568 ms. Trace:
I20260709 12:17:45.373008 18372 rpcz_store.cc:276] 0709 12:17:43.804559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.804927 (+   368us) service_pool.cc:224] Handling call
0709 12:17:43.823180 (+ 18253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.940547 (+117367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.372433 (+1431886us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400114c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:45.372592 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268468480}
I20260709 12:17:45.364023 21373 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:45.374812 21373 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.375809 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:45.376000 21373 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.376410 19041 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
W20260709 12:17:45.376878 19041 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 746) took 1550 ms (client timeout 1571 ms). Trace:
W20260709 12:17:45.377205 19041 rpcz_store.cc:269] 0709 12:17:43.826911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.375545 (+1548634us) service_pool.cc:224] Handling call
0709 12:17:45.376826 (+  1281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:45.362725 21220 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:45.352322 21321 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:17:45.352949 18370 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 545) took 1549 ms (client timeout 1930 ms). Trace:
W20260709 12:17:45.381363 18370 rpcz_store.cc:269] 0709 12:17:43.803872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.803995 (+   123us) service_pool.cc:224] Handling call
0709 12:17:45.351829 (+1547834us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400168ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.352890 (+  1061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2668717312}
W20260709 12:17:45.387027 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.393858 21372 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:45.354681 21357 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.234s	user 0.011s	sys 0.017s
I20260709 12:17:45.394412 21372 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.355898 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1636) took 2186 ms. Trace:
W20260709 12:17:45.359887 18449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2131) took 2000 ms (client timeout 2023 ms). Trace:
I20260709 12:17:45.396004 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 625) took 1933 ms. Trace:
W20260709 12:17:45.396507 18449 rpcz_store.cc:269] 0709 12:17:43.359382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.360451 (+  1069us) service_pool.cc:224] Handling call
0709 12:17:45.359688 (+1999237us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54002ab054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.359830 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051330944}
W20260709 12:17:45.397825 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.398449 21372 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:45.360424 21292 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:45.317379 21363 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:17:45.396615 18446 rpcz_store.cc:276] 0709 12:17:43.462280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.672360 (+210080us) service_pool.cc:224] Handling call
0709 12:17:43.672638 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.673219 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.673258 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.673718 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.677838 (+  4120us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:43.677875 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.677941 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.354069 (+1676128us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:45.395595 (+ 41526us) raft_consensus.cc:1685] finished
0709 12:17:45.395629 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.395922 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":47685504,"thread_start_us":3749,"threads_started":1,"wal-append.queue_time_us":251532}
I20260709 12:17:45.254729 21230 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:17:45.398737 18369 rpcz_store.cc:276] 0709 12:17:43.169942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.191433 (+ 21491us) service_pool.cc:224] Handling call
0709 12:17:43.523241 (+331808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.573752 (+ 50511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.355692 (+1781940us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54003f2350. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.355838 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762356096}
I20260709 12:17:45.299152 21367 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.241580 21222 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:45.404724 18266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.022s
W20260709 12:17:45.406594 18266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.006s	sys 0.022s
W20260709 12:17:45.406507 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 437573us
I20260709 12:17:45.409610 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1680) took 1658 ms. Trace:
W20260709 12:17:45.413568 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.036s	sys 0.085s
W20260709 12:17:45.413954 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.036s	sys 0.085s
W20260709 12:17:45.407289 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.415767 18266 connection.cc:256] RPC call timeout handler was delayed by -1.0472306409999419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:17:45.416797 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:45.413697 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 513891us
I20260709 12:17:45.437156 18375 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:45.450856 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.460263 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.462368 18263 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.657s (SENT)
W20260709 12:17:45.502314 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.413329 18297 rpcz_store.cc:276] 0709 12:17:43.751724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.779033 (+ 27309us) service_pool.cc:224] Handling call
0709 12:17:43.779524 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.779936 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.779978 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.798595 (+ 18617us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.876577 (+1077982us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:44.876614 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.876677 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.409383 (+532706us) raft_consensus.cc:1685] finished
0709 12:17:45.409403 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.409545 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":10666,"thread_start_us":1041369,"threads_started":1,"wal-append.queue_time_us":1300922}
W20260709 12:17:45.590520 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 359740us
W20260709 12:17:45.591611 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 177004us
W20260709 12:17:45.597352 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.597465 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2134) took 2231 ms (client timeout 1867 ms). Trace:
W20260709 12:17:45.598084 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.598294 18442 rpcz_store.cc:269] 0709 12:17:43.366201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.366345 (+   144us) service_pool.cc:224] Handling call
0709 12:17:45.597236 (+2230891us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54003b24d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.597376 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280178560}
I20260709 12:17:45.601032 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 541) took 1798 ms. Trace:
I20260709 12:17:45.601644 18374 rpcz_store.cc:276] 0709 12:17:43.802246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.802430 (+   184us) service_pool.cc:224] Handling call
0709 12:17:43.802797 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.957167 (+1154370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.600736 (+643569us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:45.600958 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652873984}
W20260709 12:17:45.604776 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 72e8da9e5fd9449f93381781e125fada: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:45.605278 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:45.608006 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 295041us
I20260709 12:17:45.400306 21294 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.163s
I20260709 12:17:45.319631 21255 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:45.617027 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 200355us
I20260709 12:17:45.550922 21314 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:45.619771 19037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 756) took 1721 ms (client timeout 1807 ms). Trace:
W20260709 12:17:45.620347 19037 rpcz_store.cc:269] 0709 12:17:43.899780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.937655 (+ 37875us) service_pool.cc:224] Handling call
0709 12:17:45.619531 (+1681876us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540060fed4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.619693 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324072704}
I20260709 12:17:45.624006 19038 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:17:45.623188 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 185776us
I20260709 12:17:45.629670 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2144) took 2252 ms. Trace:
I20260709 12:17:45.630234 18444 rpcz_store.cc:276] 0709 12:17:43.377449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.418839 (+ 41390us) service_pool.cc:224] Handling call
0709 12:17:43.449592 (+ 30753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.450007 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.450032 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.450655 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.893033 (+1442378us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:44.893068 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.893125 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.629395 (+736270us) raft_consensus.cc:1685] finished
0709 12:17:45.629421 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.629593 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":38189,"thread_start_us":1345644,"threads_started":1,"wal-append.queue_time_us":1610867}
I20260709 12:17:45.590860 21044 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:45.642335 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 825) took 2016 ms. Trace:
I20260709 12:17:45.610618 21375 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.618628 19036 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:45.649027 19044 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:45.653461 21383 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:45.645884 19040 rpcz_store.cc:276] 0709 12:17:43.626320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.106351 (+1480031us) service_pool.cc:224] Handling call
0709 12:17:45.155755 (+ 49404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.156056 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:45.557825 (+401769us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.641867 (+ 84042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:45.641934 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:45.642004 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.642090 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.642253 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28292,"thread_start_us":456403,"threads_started":2}
I20260709 12:17:45.610759 21339 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:17:45.619771 21372 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:17:45.662673 21377 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:45.649940 21375 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:45.676578 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 67707.8us
I20260709 12:17:45.678226 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:45.658294 21044 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:45.690531 21347 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.690941 21347 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:17:45.686718 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 267915us
I20260709 12:17:45.672403 21387 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:45.678910 21374 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.681937 21377 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.663492 21376 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:17:45.681769 21389 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:45.719125 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 101295us
I20260709 12:17:45.720463 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:45.739725 19042 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:45.708039 21387 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.717501 21292 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.714258 21295 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:45.705873 21044 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:45.752327 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:45.752931 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:45.716953 21377 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:45.753193 19006 leader_election.cc:336] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.601s (SENT)
I20260709 12:17:45.716081 21294 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:17:45.745949 21222 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:45.715559 21390 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:17:45.818607 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1706) took 2257 ms. Trace:
I20260709 12:17:45.821277 18447 rpcz_store.cc:276] 0709 12:17:43.560957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.667222 (+106265us) service_pool.cc:224] Handling call
0709 12:17:45.818337 (+2151115us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002049d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.818525 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3291332864}
I20260709 12:17:45.824316 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:45.824934 19040 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.742482 21383 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:45.715453 21374 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:45.824239 21386 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:17:45.831153 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:45.825112 21390 consensus_queue.cc:1243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:17:45.845970 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 125617us
I20260709 12:17:45.717628 21389 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:45.815220 21314 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:45.761168 21295 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
W20260709 12:17:45.874699 18441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2154) took 2220 ms (client timeout 1503 ms). Trace:
W20260709 12:17:45.875197 19006 leader_election.cc:336] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.847s (SENT)
I20260709 12:17:45.880277 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:45.757131 21292 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.875631 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:45.881331 18441 rpcz_store.cc:269] 0709 12:17:43.653953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.662843 (+  8890us) service_pool.cc:224] Handling call
0709 12:17:45.852312 (+2189469us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54003561d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.874629 (+ 22317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3495376640}
W20260709 12:17:45.855599 20973 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.003s
W20260709 12:17:45.907815 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 279813us
I20260709 12:17:45.880591 21255 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:45.927125 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:45.928068 18441 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
W20260709 12:17:45.908739 21391 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 12:17:45.926707 21341 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 2.110s	user 0.031s	sys 0.023s
I20260709 12:17:45.935985 19040 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:45.937839 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:45.938573 19040 raft_consensus.cc:2376] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5. Re-sending same reply.
W20260709 12:17:45.937461 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 89048.5us
I20260709 12:17:45.933748 21383 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.941650 21295 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:17:45.943785 21401 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:17:45.944231 21220 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 12:17:45.927606 21294 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:45.921814 21408 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.930229 21379 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.931526 21399 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:45.940343 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:45.936686 21382 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.932825 21414 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.911793 20973 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.932411 21397 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:45.954094 21380 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:17:45.967213 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:45.952301 21408 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.954247 21294 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:45.964713 21295 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:17:45.984861 19039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 745) took 2158 ms (client timeout 2071 ms). Trace:
W20260709 12:17:45.988926 19039 rpcz_store.cc:269] 0709 12:17:43.826672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.454197 (+627525us) service_pool.cc:224] Handling call
0709 12:17:45.984640 (+1530443us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540071e8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:45.984778 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319307648}
I20260709 12:17:45.985507 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1672) took 2471 ms. Trace:
I20260709 12:17:45.992168 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:45.990366 18299 rpcz_store.cc:276] 0709 12:17:43.514417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.653543 (+139126us) service_pool.cc:224] Handling call
0709 12:17:43.653801 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.940091 (+286290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.985285 (+2045194us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540021ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:45.985451 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657010432}
I20260709 12:17:45.993858 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:45.994664 19039 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:17:45.995774 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 120233us
W20260709 12:17:45.984704 21395 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:17:45.981344 21388 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:17:45.999720 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:45.997306 18297 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:17:45.985214 21402 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:17:45.997067 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:45.973261 21390 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:46.001575 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:45.979060 21409 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.983917 21396 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:45.978148 21408 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.004155 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.004369 19039 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:45.988647 21295 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.006320 21390 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:45.990692 21044 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:46.010414 21367 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.001s
W20260709 12:17:46.011051 19044 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 572) took 2425 ms (client timeout 2063 ms). Trace:
W20260709 12:17:46.013353 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.994189 21387 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.005463 19035 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:17:46.003535 21404 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:17:46.016002 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.025632 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 576) took 2179 ms. Trace:
I20260709 12:17:46.025987 19037 rpcz_store.cc:276] 0709 12:17:43.846526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.639443 (+1792917us) service_pool.cc:224] Handling call
0709 12:17:45.639934 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:45.744278 (+104344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.744306 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:45.751922 (+  7616us) log.cc:844] Serialized 52 byte log entry
0709 12:17:45.962562 (+210640us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:45.962599 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.962673 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.025403 (+ 62730us) raft_consensus.cc:1685] finished
0709 12:17:46.025421 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.025545 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19493,"raft.queue_time_us":100498,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6636,"thread_start_us":227318,"threads_started":2,"wal-append.queue_time_us":242415}
I20260709 12:17:46.027207 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 575) took 2214 ms. Trace:
I20260709 12:17:46.027777 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:46.028487 19037 raft_consensus.cc:2376] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6. Re-sending same reply.
I20260709 12:17:46.027561 19043 rpcz_store.cc:276] 0709 12:17:43.813242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.313683 (+1500441us) service_pool.cc:224] Handling call
0709 12:17:45.349098 (+ 35415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.349568 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:45.686272 (+336704us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.816144 (+129872us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.824603 (+  8459us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.824784 (+   181us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:45.824834 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.824921 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.027001 (+202080us) raft_consensus.cc:1685] finished
0709 12:17:46.027020 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.027155 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169542,"prepare.queue_time_us":358631,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":35,"raft.queue_time_us":179205,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":26241,"spinlock_wait_cycles":1682176,"thread_start_us":344150,"threads_started":2,"wal-append.queue_time_us":72238}
I20260709 12:17:46.029510 21399 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.029832 21347 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.032730 20973 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.003784 21398 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:45.967801 21382 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.039974 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2092) took 1686 ms. Trace:
I20260709 12:17:46.040462 21330 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:46.040889 21330 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:46.041326 21382 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.040396 18298 rpcz_store.cc:276] 0709 12:17:44.353652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.026957 (+673305us) service_pool.cc:224] Handling call
0709 12:17:45.110397 (+ 83440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:45.110759 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.110791 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:45.111437 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 12:17:45.427661 (+316224us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:45.675409 (+247748us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.675984 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.676055 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.039782 (+363727us) raft_consensus.cc:1685] finished
0709 12:17:46.039800 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.039916 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":254154,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":549203,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":20401,"spinlock_wait_cycles":7440768,"thread_start_us":436060,"threads_started":2,"wal-append.queue_time_us":634623}
I20260709 12:17:46.023105 21044 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.022161 21292 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.042456 21044 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.024248 21367 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.044062 21390 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.044957 21367 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:46.045130 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.018975 21295 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:46.019425 21373 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.004s
I20260709 12:17:46.018896 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:46.021497 19044 rpcz_store.cc:269] 0709 12:17:43.586055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.704707 (+118652us) service_pool.cc:224] Handling call
0709 12:17:46.010796 (+2306089us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400611054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:46.010937 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4469031680}
I20260709 12:17:45.972486 21414 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.015592 21410 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.017593 21413 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.024956 21418 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.049746 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 567) took 2585 ms. Trace:
I20260709 12:17:46.066998 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.072762 19038 rpcz_store.cc:276] 0709 12:17:43.463997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.106548 (+1642551us) service_pool.cc:224] Handling call
0709 12:17:45.623665 (+517117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.626350 (+  2685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.049577 (+423227us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540060fed0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:46.049691 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188336896}
I20260709 12:17:46.073441 18367 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:46.064198 19006 leader_election.cc:336] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.039s (SENT)
I20260709 12:17:46.050546 19040 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
W20260709 12:17:46.055392 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.98739e+06us
I20260709 12:17:46.066442 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.075219 18370 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.066048 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.076860 18369 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:46.077092 18339 connection.cc:256] RPC call timeout handler was delayed by -1.2427979910000886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 12:17:46.065436 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:46.057314 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1098326420001285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 12:17:46.077492 18339 connection.cc:256] RPC call timeout handler was delayed by -1.147393872000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260709 12:17:46.080049 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.083050 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.084170 18374 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:17:46.086426 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.047360 19039 raft_consensus.cc:2376] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7. Re-sending same reply.
W20260709 12:17:46.087656 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.058s (SENT)
I20260709 12:17:46.090934 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.093142 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.093451 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.094156 18371 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.093150 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.095326 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.096068 18299 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:46.051602 21346 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.653s	user 0.016s	sys 0.021s
I20260709 12:17:46.110778 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.025091 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 749) took 2197 ms. Trace:
I20260709 12:17:46.100135 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.115160 19041 rpcz_store.cc:276] 0709 12:17:43.828141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.377791 (+1549650us) service_pool.cc:224] Handling call
0709 12:17:45.378115 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.378496 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.378521 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.378963 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.667453 (+288490us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:45.667507 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.667601 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.980787 (+313186us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:46.023024 (+ 42237us) raft_consensus.cc:1685] finished
0709 12:17:46.023056 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.025007 (+  1951us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55376,"raft.queue_time_us":33017,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":17276,"spinlock_wait_cycles":6815616,"thread_start_us":288075,"threads_started":1,"wal-append.queue_time_us":449181}
I20260709 12:17:46.058203 21269 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.122059 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.054332 21191 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.014758 21396 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.100646 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.060937 21389 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:46.059149 21365 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.112287 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.151392 18443 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:46.152364 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.112991 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.153362 18448 raft_consensus.cc:2376] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9. Re-sending same reply.
I20260709 12:17:45.996601 21411 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:46.114698 18446 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:17:46.105396 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.110339 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.015122 21409 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.120500 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:46.202865 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 82475.5us
I20260709 12:17:46.203780 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:46.121412 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.059386 21374 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.116883 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.052451 20998 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:46.058038 21271 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.057518 21278 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.054775 21237 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.048084 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.219635 18441 raft_consensus.cc:2376] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7. Re-sending same reply.
I20260709 12:17:46.056444 21323 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:46.049249 21044 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:46.049896 21414 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.048589 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 549) took 2240 ms. Trace:
I20260709 12:17:45.992220 21397 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.100216 21273 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.145519 21269 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.117194 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:46.081398 21299 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.056214 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 751) took 2227 ms. Trace:
I20260709 12:17:46.256664 19036 rpcz_store.cc:276] 0709 12:17:43.829014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.380263 (+1551249us) service_pool.cc:224] Handling call
0709 12:17:45.380584 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.658150 (+277566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.055960 (+397810us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400564fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:46.056155 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391329024}
I20260709 12:17:46.258425 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:46.259696 18372 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:46.262022 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.262832 18372 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:46.132570 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.133934 18298 raft_consensus.cc:2410] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:46.132795 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.132938 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.264109 18449 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:46.122711 19044 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:46.131608 18369 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:46.130527 18370 raft_consensus.cc:2410] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:46.139552 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:46.063310 19035 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 758) took 2152 ms (client timeout 1657 ms). Trace:
I20260709 12:17:46.145800 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:46.145913 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.267482 18447 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:46.089704 21347 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:46.173017 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.180764 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:46.109361 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.200001 18442 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:46.023619 21294 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:46.207959 19006 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 3.027s (SENT) [suppressed 73 similar messages]
W20260709 12:17:46.278527 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 70601.1us
I20260709 12:17:46.121842 21219 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.041800 21330 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:46.229046 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 202410us
W20260709 12:17:46.279702 19006 leader_election.cc:336] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.240s (SENT)
I20260709 12:17:46.280350 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:46.281322 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.281499 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:46.088141 21418 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.232576 18375 rpcz_store.cc:276] 0709 12:17:43.808500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.823043 (+ 14543us) service_pool.cc:224] Handling call
0709 12:17:43.823311 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.437739 (+1614428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.048384 (+610645us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400394c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:46.048536 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3673387904}
I20260709 12:17:46.122460 21417 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.054477 21168 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:46.299096 21168 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.223163 21271 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.301926 21418 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:46.303483 21413 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:46.227682 21237 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.214762 21314 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.306022 21314 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.306406 21314 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.306315 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.307391 20998 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.232839 21323 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.310703 21323 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.218441 21210 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:46.312443 21210 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.314582 21210 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.151975 21191 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.315541 21330 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.316507 21191 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.087924 21410 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.242362 18441 pending_rounds.cc:85] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 2
I20260709 12:17:46.317238 18441 pending_rounds.cc:107] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:46.317523 18441 raft_consensus.cc:2889] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:46.248463 19041 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:46.250073 21397 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:46.204147 21411 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.280706 19036 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:46.051911 21322 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:46.264197 18440 raft_consensus.cc:2376] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6. Re-sending same reply.
I20260709 12:17:46.271958 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:46.322336 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.273144 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.277254 21041 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:46.208853 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:46.282950 18369 raft_consensus.cc:2376] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5. Re-sending same reply.
I20260709 12:17:46.254952 21269 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.242740 21209 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.198319 21371 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.305603 21237 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.227841 21278 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.255669 21299 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.221946 21377 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.341782 21191 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:46.312014 21219 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:46.303033 21271 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:46.328650 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 119880us
W20260709 12:17:46.268982 19035 rpcz_store.cc:269] 0709 12:17:43.911093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.922643 (+ 11550us) service_pool.cc:224] Handling call
0709 12:17:46.063063 (+2140420us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003a0554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:46.063222 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537494272}
I20260709 12:17:46.330083 18440 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:46.333016 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53363.7us
I20260709 12:17:46.201207 21365 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.318857 21410 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.343354 19044 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:46.319546 21373 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:46.321719 21222 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:46.366396 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 286991us
W20260709 12:17:46.414247 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 331800us
W20260709 12:17:46.326224 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.417869 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:46.326632 18371 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:46.355365 21413 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:46.378368 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:46.375588 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.418993 18442 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:46.421088 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.421957 18370 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:46.423287 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.424997 18371 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:46.427510 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49198.2us
I20260709 12:17:46.430548 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:46.433279 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:46.434830 18338 leader_election.cc:336] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.536s (SENT)
W20260709 12:17:46.436710 18338 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.701s (SENT)
I20260709 12:17:46.437079 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:46.438325 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:46.366223 21191 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.442934 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.451517 18445 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:46.443833 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.452224 18440 raft_consensus.cc:2376] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5. Re-sending same reply.
I20260709 12:17:46.355634 21371 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.445524 18366 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:46.463583 18373 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:46.477003 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.477747 18445 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:46.454795 21333 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:46.460825 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:46.442412 21418 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:46.480134 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.460186 21191 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.414204 21376 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:17:46.438289 21241 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:17:46.488076 19037 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:46.461408 21269 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:46.471581 21374 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.487749 21387 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:46.440296 21044 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:17:46.482343 21333 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.488740 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.424654 21408 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.516228 21278 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.508455 21389 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:17:46.517228 21371 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.517776 21409 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.461248 21397 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:46.453168 21419 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:17:46.438621 21413 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.472015 21414 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:17:46.493069 21367 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.429261 21322 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.490844 21314 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.523072 21399 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.519439 21374 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:46.576073 21374 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.577744 21413 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:46.524858 21423 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:17:46.524029 21435 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.578631 21367 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:46.511384 21382 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:17:46.579807 21382 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:46.579694 21367 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.578872 21435 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:46.582248 21426 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:17:46.582475 21435 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.572037 21419 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:17:46.584128 21431 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:17:46.585531 21427 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:17:46.529075 21333 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:46.660521 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:46.669849 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 233016us
W20260709 12:17:46.449065 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.023s (SENT)
W20260709 12:17:46.814692 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 365490us
I20260709 12:17:46.673460 18366 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:17:46.815614 21378 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.816888 21378 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.817785 21365 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.771939 21292 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:17:46.819383 21403 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:46.824663 21398 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.825624 21398 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.826205 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.827265 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.828182 18297 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:46.830430 18366 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:17:46.830504 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:46.679193 18412 leader_election.cc:336] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.584s (SENT)
I20260709 12:17:46.832680 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.833750 18375 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.679009 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.481889 18372 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.835300 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.691200 18449 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:46.692549 18447 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:46.704315 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1646) took 3304 ms. Trace:
W20260709 12:17:46.705005 19006 leader_election.cc:336] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.703s (SENT)
I20260709 12:17:46.837635 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:46.710475 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.710248 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.491792 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.711452 19036 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:46.867182 18197 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:46.868709 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.711453 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.713271 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.714406 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.715063 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.718200 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:46.722162 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 9fc8b80684174ecb8c933560e8143c49: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:46.726553 18369 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:46.728276 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 759) took 2806 ms. Trace:
I20260709 12:17:46.729588 18367 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:46.730099 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.732359 19041 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:46.517592 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.872067 19039 rpcz_store.cc:276] 0709 12:17:43.921667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.307270 (+2385603us) service_pool.cc:224] Handling call
0709 12:17:46.307623 (+   353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:46.307961 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.307991 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:46.308570 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:17:46.389252 (+ 80682us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:46.489097 (+ 99845us) log.cc:844] Serialized 14 byte log entry
0709 12:17:46.489379 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.489474 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.596782 (+107308us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b18002172a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:46.727511 (+130729us) raft_consensus.cc:1685] finished
0709 12:17:46.727709 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.728037 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153884,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":20366,"raft.queue_time_us":4115,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":19601,"spinlock_wait_cycles":36478848,"thread_start_us":134745,"threads_started":2,"wal-append.queue_time_us":183370}
I20260709 12:17:46.875654 18444 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.733871 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 0) took 2920 ms. Trace:
I20260709 12:17:46.876521 19042 rpcz_store.cc:276] 0709 12:17:43.823959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.091540 (+1267581us) service_pool.cc:224] Handling call
0709 12:17:46.732158 (+1640618us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b4c00280b80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6711f644 00007f3c67146f2d 00007f3c67146b93 00007f3c68e76c92 00007f3c68e7684a 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691
0709 12:17:46.733532 (+  1374us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540066e4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:46.733649 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442194432}
I20260709 12:17:46.736313 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:46.878142 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 577) took 1003 ms. Trace:
I20260709 12:17:46.878484 19043 rpcz_store.cc:276] 0709 12:17:45.874543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.029451 (+154908us) service_pool.cc:224] Handling call
0709 12:17:46.735988 (+706537us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:46.878088 (+142100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161871104}
I20260709 12:17:46.738296 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:46.748777 20998 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:46.880404 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 142450us
I20260709 12:17:46.882179 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.882548 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:46.762223 21209 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:46.883531 21209 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.883692 21409 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.884281 21409 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.884677 21409 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.886281 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.887079 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.888200 18300 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:46.887997 20973 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:46.889010 20973 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:17:46.889770 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.429903 21222 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.890841 18294 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:46.891739 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.892481 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.893666 18291 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.453886 21227 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:17:46.515002 21269 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:46.906092 21269 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.907562 21044 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:46.908495 21044 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.763737 21314 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.764148 21294 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.911146 21294 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.913779 21294 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.764467 21219 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.765475 21299 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.546754 21295 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.588714 21374 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.595537 21375 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.922401 21445 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:46.923744 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 86017.9us
I20260709 12:17:46.923532 21445 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.925038 20946 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:46.925623 20946 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:46.598073 21377 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.927783 21220 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:46.580185 21382 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:46.930593 21382 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.837450 18368 rpcz_store.cc:276] 0709 12:17:43.399955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.400089 (+   134us) service_pool.cc:224] Handling call
0709 12:17:43.421060 (+ 20971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.647986 (+226926us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:43.812007 (+164021us) log.cc:844] Serialized 14 byte log entry
0709 12:17:43.909811 (+ 97804us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.364422 (+1454611us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b18003af780. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1
0709 12:17:45.365865 (+  1443us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b18003af780. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:46.034990 (+669125us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b18003af780. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:17:46.352784 (+317794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:46.352837 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:46.352884 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.703621 (+350737us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:46.703738 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.704053 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":845946,"prepare.queue_time_us":162469,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":5113,"raft.queue_time_us":2183836,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":200,"spinlock_wait_cycles":625378816,"thread_start_us":364474,"threads_started":2,"wal-append.queue_time_us":1135623,"wal-append.run_cpu_time_us":1955,"wal-append.run_wall_time_us":1046597}
I20260709 12:17:46.932331 21382 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.596342 21387 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:46.592826 21396 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.591965 21408 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.933691 21387 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:46.919348 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.544773 21418 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:46.934629 21418 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:46.483165 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:46.833814 18298 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:46.833967 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 155070us
I20260709 12:17:46.935948 21418 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.937343 21418 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.873502 21241 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.939661 21241 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.942708 21241 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.915858 21044 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.943693 21044 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.944294 21044 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.946642 21044 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.947059 21295 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.947961 21435 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:46.948611 21435 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:46.949687 21435 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.874902 21230 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.953017 21191 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.876201 19044 raft_consensus.cc:2368] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:46.885190 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:46.956341 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 71393.3us
I20260709 12:17:46.956677 21295 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.957409 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.935006 21387 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.959249 21387 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.959532 21408 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.901777 21378 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.961030 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:46.915619 18448 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:46.916224 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.918735 18296 raft_consensus.cc:2368] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:17:46.962630 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:46.963496 21378 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:46.963869 21378 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:46.964288 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:46.964468 21408 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.964625 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.964763 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:46.965404 18448 raft_consensus.cc:2376] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6. Re-sending same reply.
I20260709 12:17:46.965636 18446 raft_consensus.cc:2368] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:17:46.915162 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.921464 21227 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:46.966873 21227 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.967362 21227 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:46.967633 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:46.957413 21330 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.008s
I20260709 12:17:46.914288 21398 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.969183 21408 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.969507 21237 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.970216 21237 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.971041 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:46.961318 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.971889 21295 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.573856 21411 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:46.965595 21378 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.600883 21417 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.922508 21374 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:46.902671 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 197981us
I20260709 12:17:46.932457 20946 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.976157 20946 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:46.979759 21295 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:46.980386 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:46.979427 21407 consensus_queue.cc:1175] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "177254018ba7448aa2b9110c9c9be9e3" responder_term: 7 status { last_received { term: 7 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 7 index: 7 } }
I20260709 12:17:46.980875 21295 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:46.981479 21417 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.979826 21222 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:46.983458 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:46.983903 21417 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:46.984453 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.985127 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:46.984443 21222 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.986326 21299 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:46.986406 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.986905 21299 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:46.987327 18298 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:46.987473 21299 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.988777 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.989603 18298 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:46.988811 18292 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:46.985682 18296 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:46.981475 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:46.992918 21377 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.993172 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.993746 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:46.985070 21417 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.994436 18292 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:46.995127 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:46.994496 18296 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:46.924407 18441 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:46.993172 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.002916 18299 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:47.004294 18291 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:47.015470 21377 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:47.016451 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.016676 21377 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:46.870556 18197 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 5, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:47.024614 18444 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:47.004540 21237 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.004558 21445 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:47.036816 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.034861 18298 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:47.023257 21191 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.005843 21375 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.040410 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.030362 20998 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.053728 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.064105 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.068305 18441 raft_consensus.cc:2368] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:17:47.064968 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.066781 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:47.067740 18299 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 4.
W20260709 12:17:47.082707 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.085214 18443 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:47.086318 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.087136 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.087648 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.087999 18446 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:47.088327 18447 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:47.088766 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.088935 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.089388 18449 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:47.089561 18446 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:47.091928 21390 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.093953 20973 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:47.088024 18442 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:47.095954 18197 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 5, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 5 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:47.097915 18197 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.099325 18368 pending_rounds.cc:85] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 7
I20260709 12:17:47.099682 18368 pending_rounds.cc:107] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:17:47.099715 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.100744 18444 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:47.104441 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.105290 18448 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.119202 18446 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:17:47.121613 21209 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:47.126119 18194 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 292) took 12059 ms (client timeout 15000 ms). Trace:
W20260709 12:17:47.115779 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69571.9us
I20260709 12:17:47.127820 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.128607 18374 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:47.128815 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:47.129508 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54112.3us
I20260709 12:17:47.071599 21241 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:17:47.116736 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.135329 18299 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:47.100042 18368 raft_consensus.cc:2889] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:47.115831 21269 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.131333 21399 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.141078 21269 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.140987 21399 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:47.137152 18298 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:47.147912 18448 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:47.150682 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.152161 21230 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:17:47.140926 21237 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.148586 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:47.158049 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.147354 21209 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.164284 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.170708 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:47.176342 21347 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.903s	user 0.010s	sys 0.006s
I20260709 12:17:47.176429 21333 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.177318 19040 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:47.178872 21333 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:47.179554 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 131897us
W20260709 12:17:47.126850 18194 rpcz_store.cc:269] 0709 12:17:35.066058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.066308 (+   250us) service_pool.cc:224] Handling call
0709 12:17:47.125802 (+12059494us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:43.046535 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:43.060505 (+ 13970us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:43.060547 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:17:43.060566 (+    19us) tablet.cc:662] Decoding operations
0709 12:17:43.062442 (+  1876us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:43.062500 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:43.062535 (+    35us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:17:43.119852 (+ 57317us) tablet.cc:701] Row locks acquired
0709 12:17:43.119869 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:17:43.125445 (+  5576us) write_op.cc:270] Start()
0709 12:17:43.138695 (+ 13250us) write_op.cc:276] Timestamp: P: 1783599463124912 usec, L: 0
0709 12:17:43.138743 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:17:43.150667 (+ 11924us) log.cc:844] Serialized 9406 byte log entry
0709 12:17:44.024862 (+874195us) op_driver.cc:464] REPLICATION: finished
0709 12:17:45.007858 (+982996us) write_op.cc:301] APPLY: starting
0709 12:17:45.007953 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:45.008506 (+   553us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:45.020137 (+ 11631us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:45.361368 (+341231us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:45.361731 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 12:17:45.361773 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:45.362165 (+   392us) log.cc:844] Serialized 164 byte log entry
0709 12:17:45.632434 (+270269us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:45.657169 (+ 24735us) write_op.cc:454] Released schema lock
0709 12:17:45.667004 (+  9835us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:45.708980 (+ 41976us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:45.712102 (+  3122us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":222405,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":961895,"apply.run_cpu_time_us":82099,"apply.run_wall_time_us":674621,"num_ops":24,"prepare.queue_time_us":530013,"prepare.run_cpu_time_us":25181,"prepare.run_wall_time_us":314033,"raft.queue_time_us":424137,"raft.run_cpu_time_us":3690,"raft.run_wall_time_us":43740,"replication_time_us":885650,"thread_start_us":1093181,"threads_started":5,"wal-append.queue_time_us":552114,"wal-append.run_cpu_time_us":34798,"wal-append.run_wall_time_us":2652793}]]}
I20260709 12:17:47.181377 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.168234 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.182932 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.184378 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:47.190532 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.191630 18298 raft_consensus.cc:2410] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:47.193296 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.193195 21227 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.194872 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.196249 18446 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:47.195487 18441 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:47.196712 21269 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:17:47.199216 21390 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:47.199944 18440 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:47.199586 18197 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 7 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.200538 21330 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.200974 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:47.200841 21409 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.160168 21237 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:47.218798 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.221535 18446 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:47.219502 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.148012 20973 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:17:47.223047 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53812.4us
I20260709 12:17:47.223827 18442 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:47.228412 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:47.225927 18299 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 4.
I20260709 12:17:47.232178 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.226275 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:47.230830 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.235030 18296 raft_consensus.cc:2410] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:47.236202 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:47.223870 18197 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 7 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.233217 21295 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.238392 19044 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:17:47.239274 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.239513 21420 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:47.240032 21420 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:47.240087 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:47.240367 21295 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:47.240775 19041 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:47.240841 21295 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.240994 21420 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.201536 21392 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.243212 18442 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:17:47.243901 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 85f32970b7374dcf9b9bac47791c71e3: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:47.244488 21378 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.245100 21409 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:47.245996 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.247556 18293 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:47.247826 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.247939 21408 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.248577 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:47.249348 18371 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.195458 21399 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.243223 21392 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.216620 21269 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:47.235807 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.254565 19040 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 4.
I20260709 12:17:47.254979 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:47.224220 21237 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.257820 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.258700 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.224428 21330 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.259692 18291 raft_consensus.cc:2368] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7.
W20260709 12:17:47.259992 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 89137.5us
I20260709 12:17:47.260761 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.261508 18297 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.261874 21330 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.262169 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:47.262866 21269 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:17:47.263689 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.264482 18295 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:47.264871 21392 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:47.265379 21220 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:47.265998 21396 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:17:47.266420 21220 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:47.266891 21220 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:47.221246 21241 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:47.267928 21220 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.269204 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:47.269609 21220 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.270651 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.262919 20946 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:47.268348 21241 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.273061 18440 raft_consensus.cc:2368] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:47.274163 18447 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:47.278079 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.279556 18371 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:47.247043 21378 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.286074 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.287184 18298 raft_consensus.cc:2410] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:47.277557 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:47.199260 21299 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.288964 18369 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:47.289250 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:47.291280 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.294631 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.298035 18441 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:47.296691 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:47.281752 21390 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:47.308888 18294 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:47.307129 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:47.291165 21396 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.307163 19040 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:47.308405 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:47.313251 18299 raft_consensus.cc:2368] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 4. Current term is 5.
I20260709 12:17:47.314766 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:47.296756 21227 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:47.351657 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49005.3us
I20260709 12:17:47.239122 21398 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:17:47.356699 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:47.303133 21230 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.321404 21333 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:47.335609 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55299.6us
I20260709 12:17:47.339514 18370 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:47.340492 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.341795 21417 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:47.343271 21299 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:47.343919 21453 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:47.343693 21330 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:47.306906 18293 raft_consensus.cc:2376] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
W20260709 12:17:47.358692 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56532us
I20260709 12:17:47.354535 18197 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 5, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:47.358814 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.357879 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.318337 20946 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:47.375959 21431 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:17:47.376528 21333 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:47.380761 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.364223 18449 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:47.311040 21390 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:47.383594 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:47.401453 18297 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:47.378074 21417 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:47.381052 21230 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:47.428941 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.453285 18299 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:47.430254 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:47.455374 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:47.434068 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.456453 18297 raft_consensus.cc:2376] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
I20260709 12:17:47.432693 18197 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.409363 18369 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:17:47.365047 21396 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.429118 21408 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:47.429039 21382 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.469513 18197 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 6, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:47.478924 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.465837 21398 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.464352 21377 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.482956 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.527387 21377 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:47.484992 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 144633us
I20260709 12:17:47.485790 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.482442 21241 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:47.486570 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.482717 21398 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:47.486745 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.529078 21398 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.472601 21408 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.530598 21408 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.487417 21422 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:47.486779 21396 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.485049 21417 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.490911 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:47.491559 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:47.491762 21230 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.493831 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:47.535180 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.535806 18369 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:47.535173 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:47.503722 21390 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.507501 21333 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.511212 21295 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.429798 20946 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.521075 21237 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:47.527796 21377 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.528533 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:47.536840 21191 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:47.483578 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.532998 18197 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:47.542624 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.554543 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.538870 21241 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:47.555229 18295 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:47.556743 18299 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:47.548921 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:47.553373 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.557849 18294 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.558656 18291 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 12:17:47.559003 18291 pending_rounds.cc:85] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 7
I20260709 12:17:47.559257 18291 pending_rounds.cc:107] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:17:47.559474 18291 raft_consensus.cc:2889] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:47.553440 21382 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:47.553591 21417 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.562258 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.543543 21375 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:17:47.531497 20998 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:47.552543 18298 raft_consensus.cc:2376] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
I20260709 12:17:47.565831 18296 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:47.562950 18293 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:47.556242 21241 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.575707 18291 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:17:47.562366 21382 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:47.573464 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 257530us
I20260709 12:17:47.571713 21191 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.591005 18197 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 3, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 3 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.561141 21269 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.572263 21375 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:47.578985 21241 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.585798 21382 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.596140 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:47.611722 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 270005us
I20260709 12:17:47.614655 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.617976 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:47.619436 18291 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:47.620436 21269 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:47.624722 18375 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
W20260709 12:17:47.624853 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52119.9us
I20260709 12:17:47.622944 20946 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.630980 18449 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:47.634404 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:47.635134 18299 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.641621 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:47.641971 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.642719 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:47.663807 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.672473 18197 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 5, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:17:47.754081 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:47.866271 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.680452 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:47.868173 18442 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:17:47.675398 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91051.4us
I20260709 12:17:47.693771 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:47.702077 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.969s (SENT)
W20260709 12:17:47.886716 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 255026us
W20260709 12:17:47.886814 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 353243us
W20260709 12:17:47.888888 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 186837us
I20260709 12:17:47.889402 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:47.787209 18197 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.891155 18197 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 2 to 5, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 5 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:47.749264 18294 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:47.887209 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:47.873104 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:47.670401 20946 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.794704 21396 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.884994 21237 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:47.726045 21295 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:47.700881 21375 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:47.910101 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.206878 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2175) took 1032 ms. Trace:
I20260709 12:17:48.144521 19044 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:48.227676 18369 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:47.960757 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 173598us
W20260709 12:17:47.989392 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84169.3us
I20260709 12:17:48.128473 18299 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:48.125655 18441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.016s
W20260709 12:17:48.231860 18441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.014s	sys 0.017s
W20260709 12:17:48.225812 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 336436us
I20260709 12:17:48.001855 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.001412 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.241254 18373 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:48.004528 18197 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 4, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:48.227551 18446 rpcz_store.cc:276] 0709 12:17:47.174060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.233696 (+ 59636us) service_pool.cc:224] Handling call
0709 12:17:47.233975 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.255550 (+ 21575us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:47.570170 (+314620us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.206454 (+636284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:48.206522 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:48.206588 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.206692 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:48.206807 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32833,"prepare.queue_time_us":34235,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":28359,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":56923,"thread_start_us":868662,"threads_started":3,"wal-append.queue_time_us":614104}
W20260709 12:17:47.959594 21447 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:17:47.901515 21191 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:48.233562 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.255807 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:47.909499 21378 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:48.264219 18374 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.112533 21374 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.235991 20946 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:48.254964 21377 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.687s	user 0.002s	sys 0.008s
I20260709 12:17:48.289321 18372 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:48.292193 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:48.290845 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:48.260541 18445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.026s	sys 0.070s
W20260709 12:17:48.297140 18445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.026s	sys 0.070s
I20260709 12:17:48.293342 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.298197 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.289985 21299 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:17:48.295925 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 511267us
I20260709 12:17:48.302556 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.303321 18370 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.303397 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.270406 21237 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.313047 18373 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:48.281050 20946 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.308947 18197 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:48.333127 18372 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:48.339764 21482 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.002s
I20260709 12:17:48.259768 21396 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.350348 18370 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:48.351998 21470 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.352417 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.352619 21470 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:48.353314 18373 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.361456 21396 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:48.374104 21398 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.844s	user 0.004s	sys 0.006s
I20260709 12:17:48.333309 21295 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:17:48.378216 18374 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:48.379876 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:48.380971 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 46385.5us
I20260709 12:17:48.282141 21374 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.355011 21470 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:48.297608 21377 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.397670 18370 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:48.404004 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:48.381110 21408 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.004s
I20260709 12:17:48.406910 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.401469 21220 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.422493 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:48.424535 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 165961us
I20260709 12:17:48.431960 18373 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:48.377578 21398 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.437968 21398 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:48.438844 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.441442 18372 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:48.443810 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.444743 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.445050 18372 raft_consensus.cc:2376] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6. Re-sending same reply.
I20260709 12:17:48.457324 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.441378 21392 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.466311 18370 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:48.442433 21396 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.407056 21408 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:48.485520 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.485327 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.486248 18374 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.433000 21220 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.491812 18373 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:48.487438 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.494302 18297 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.494282 18372 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.496924 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:48.501155 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 228091us
I20260709 12:17:48.501657 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 579) took 1153 ms. Trace:
I20260709 12:17:48.502056 18375 rpcz_store.cc:276] 0709 12:17:47.345544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.345712 (+   168us) service_pool.cc:224] Handling call
0709 12:17:47.346089 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.628092 (+282003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.498249 (+870157us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54000f9454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:48.498399 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640355072}
I20260709 12:17:48.503363 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.504098 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.504882 18374 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:48.505878 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.506135 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:48.503036 21396 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.511659 21220 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:48.512066 21396 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:48.512737 21494 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:17:48.517781 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.519866 18372 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:48.520154 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.525419 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.526396 18374 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.482892 21392 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:48.529050 21241 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.904s	user 0.007s	sys 0.004s
I20260709 12:17:48.530759 21241 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.533221 21241 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:48.534458 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.535065 18366 raft_consensus.cc:2376] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6. Re-sending same reply.
I20260709 12:17:48.540870 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.541807 18370 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.544064 18373 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:17:48.537007 21374 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:48.544934 21396 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:48.548660 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45749.6us
I20260709 12:17:48.528966 21333 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:48.552448 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.553397 20973 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:17:48.554206 21497 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.554451 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.554771 20973 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:17:48.554735 21497 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.556524 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.558450 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.559566 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.561023 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.566082 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.567642 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.570660 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.571003 21392 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:48.571201 18369 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.572110 21491 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:48.574167 21497 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:48.575352 18339 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.232s (SENT)
I20260709 12:17:48.575876 18371 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:48.576303 21408 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:48.578620 18370 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:48.552392 21333 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.583894 21333 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:48.579375 21382 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.909s	user 0.011s	sys 0.000s
I20260709 12:17:48.587536 21382 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.589807 21382 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:48.558192 20973 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:48.591257 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.592020 20973 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.593186 20973 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:17:48.594018 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.594805 18299 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:48.595350 18366 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.596813 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.597652 18293 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 4.
I20260709 12:17:48.599247 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.550316 21377 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:48.613348 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.614797 18366 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:48.615775 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.615912 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.615993 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.616597 18366 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:48.617203 18369 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:48.617462 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.618634 18368 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:48.619774 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.560667 21241 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.620451 18370 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:48.622911 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:48.623054 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:48.623641 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:48.625280 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 75599.5us
I20260709 12:17:48.625380 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.625293 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.626451 18369 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:48.627476 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.628007 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.628582 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.629449 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:48.629690 18366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:48.628544 18373 raft_consensus.cc:2376] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7. Re-sending same reply.
I20260709 12:17:48.631505 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:48.632601 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.633106 18443 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:48.633349 18445 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:48.633368 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.633625 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.634613 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:48.634783 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:48.635396 19044 raft_consensus.cc:2376] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6. Re-sending same reply.
I20260709 12:17:48.635329 20946 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.630403 18366 raft_consensus.cc:606] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.620714 21382 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:48.636566 21382 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:48.637527 18366 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:48.637579 20946 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:48.638634 21382 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:48.638800 18366 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:48.639540 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:48.650549 18449 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:48.650599 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.651997 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.651206 18449 pending_rounds.cc:85] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:17:48.653163 18369 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:48.667302 18370 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:48.667668 18370 pending_rounds.cc:85] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 2
I20260709 12:17:48.667874 18370 pending_rounds.cc:107] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.3
W20260709 12:17:48.665969 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 96723.5us
I20260709 12:17:48.667153 18449 pending_rounds.cc:107] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:48.668087 18370 raft_consensus.cc:2889] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:48.669440 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.669780 18373 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:17:48.670253 18299 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:48.671377 18291 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:48.668495 18449 raft_consensus.cc:2889] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:48.671970 18369 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:17:48.670378 18373 pending_rounds.cc:85] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:17:48.703936 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.697718 18375 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.676483 18448 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:17:48.739039 18299 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:48.705045 18294 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:48.704320 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.689070 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 502) took 1149 ms. Trace:
I20260709 12:17:48.711836 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.740733 18293 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:48.707106 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.666045 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.723188 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:48.689772 18374 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:48.700415 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:48.743336 18373 pending_rounds.cc:107] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:17:48.688119 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.700274 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:48.743944 18373 raft_consensus.cc:2889] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:48.718052 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:48.744410 18367 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:48.707839 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.744839 19041 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:48.708204 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.722122 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:48.709017 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:48.736780 19040 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:48.744400 18370 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:48.747182 19044 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:48.708601 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:48.740571 18297 rpcz_store.cc:276] 0709 12:17:47.539377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.539532 (+   155us) service_pool.cc:224] Handling call
0709 12:17:48.688855 (+1149323us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b540037f0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:48.688988 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160022400}
W20260709 12:17:48.760104 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59795.6us
I20260709 12:17:48.760792 18300 raft_consensus.cc:2368] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:48.655853 21408 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.781049 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:48.781879 18369 raft_consensus.cc:606] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.782299 18369 raft_consensus.cc:614] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:48.742955 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:48.760905 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:48.779798 18296 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:48.783969 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:48.673027 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:48.784652 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 153281us
I20260709 12:17:48.784837 18292 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
W20260709 12:17:48.789098 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46318.2us
I20260709 12:17:48.802490 18291 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:48.803128 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 650) took 1174 ms. Trace:
I20260709 12:17:48.788617 18298 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:48.687618 20946 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.716570 21377 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.672282 21398 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:48.819759 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:48.809702 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.813270 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:48.821595 18299 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.816897 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:48.817135 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:48.788985 21230 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:48.827203 18300 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.827575 19038 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:48.823196 18296 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.824126 18375 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:48.825625 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:48.816412 18295 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:48.826344 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:48.807337 21241 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:48.833779 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 111749us
I20260709 12:17:48.839105 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:48.803689 18441 rpcz_store.cc:276] 0709 12:17:47.613315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.613425 (+   110us) service_pool.cc:224] Handling call
0709 12:17:47.613766 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.618190 (+  4424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.618219 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:47.618712 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:17:48.232433 (+613721us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:48.264488 (+ 32055us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.567409 (+302921us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18003b9f20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:17:48.760579 (+193170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.760814 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:48.760876 (+    62us) raft_consensus.cc:1685] finished
0709 12:17:48.787419 (+ 26543us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:48.787597 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4829,"prepare.queue_time_us":32803,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":309564,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":202,"spinlock_wait_cycles":2665856,"thread_start_us":806108,"threads_started":2,"wal-append.queue_time_us":795001}
W20260709 12:17:48.836560 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50768.1us
I20260709 12:17:48.812739 21470 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.789104 21417 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:17:48.699105 21333 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.844022 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:48.818140 21396 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:48.848433 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:48.849618 18294 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:48.870489 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.837558 21241 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.816831 21408 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.879680 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:48.846505 21230 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:48.870641 21390 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:48.880419 21237 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:48.870420 21399 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.887712 21507 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:48.888397 21496 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:17:48.889003 21508 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:48.889030 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:48.891497 21299 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.895401 21398 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.895892 21295 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.898633 21470 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:48.902202 21333 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:48.906256 18372 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:17:48.908349 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:48.970542 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:48.971238 18371 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.880666 21241 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.912592 21414 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.913235 18441 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:48.860703 21396 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.915059 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:48.915477 21408 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:48.916654 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:48.926777 19043 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:48.978520 18446 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:48.930260 21390 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:48.863348 21377 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:48.938472 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:48.950383 18299 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:48.975675 19041 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
W20260709 12:17:48.986307 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 69710.3us
I20260709 12:17:48.966452 21237 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:48.989804 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:48.989552 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 105728us
I20260709 12:17:48.966624 21299 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:48.990830 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 117331us
I20260709 12:17:48.991540 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:48.993251 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:48.993822 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 598827us
I20260709 12:17:48.975687 20946 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:48.980080 21396 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:49.004848 18291 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.996956 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:49.004105 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:49.012749 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:49.040668 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49249.3us
I20260709 12:17:49.040680 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:48.979610 21508 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:49.036770 18440 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:48.981555 21507 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:48.990104 21497 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.993530 21237 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:48.975031 21295 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:49.055591 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 166644us
I20260709 12:17:48.980197 20973 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:17:48.996409 21299 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.975304 21398 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:48.996574 21382 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:49.080318 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 89446us
I20260709 12:17:48.982023 21414 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.069286 21509 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.037585 21333 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.050658 21392 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:49.185348 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 107163us
I20260709 12:17:49.208712 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:49.185501 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 288207us
I20260709 12:17:49.180495 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:49.181098 18338 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:49.216349 18338 leader_election.cc:403] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.189181 18371 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:49.082029 21497 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.207360 18291 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:49.210266 21522 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.218288 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:49.196022 21382 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.209641 21333 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.202041 21509 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:49.226708 21399 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:49.235400 21382 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.152786 21230 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.226307 21520 raft_consensus.cc:2764] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:49.231701 20946 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.243145 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:49.201972 21408 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:17:49.228678 21522 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.233673 21333 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.153075 21390 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.201498 21330 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:17:49.241170 21347 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:49.239684 21509 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:49.267246 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 226662us
I20260709 12:17:49.245729 21374 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.253154 20946 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.257359 21522 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:49.269652 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 190124us
I20260709 12:17:49.324201 21470 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:49.324299 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:49.327150 21515 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.093s	user 0.003s	sys 0.000s
W20260709 12:17:49.316545 21516 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:17:49.336558 21519 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:17:49.316851 21498 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:17:49.268925 21390 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.309804 20946 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:49.319603 21510 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:17:49.240839 21507 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.242303 21508 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:49.321324 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.242690 21399 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:17:49.327580 21374 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.268249 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:49.271088 21392 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:49.340919 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 258492us
I20260709 12:17:49.318337 21517 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:49.309996 21377 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:49.341068 21521 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.131s	user 0.000s	sys 0.006s
W20260709 12:17:49.352988 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 272688us
W20260709 12:17:49.364766 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.351647 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 135146us
I20260709 12:17:49.382941 21398 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.383044 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:49.385902 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:49.387598 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119463us
I20260709 12:17:49.393651 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:17:49.389593 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.403079 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:49.389952 21295 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.404109 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:49.350338 21392 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:49.393519 21377 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:49.488246 21525 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:17:49.489274 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.388198 21517 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:49.532791 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 208145us
W20260709 12:17:49.533205 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 139623us
I20260709 12:17:49.486670 21527 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:17:49.536566 21392 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.530138 21532 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:49.532498 21526 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.585675 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:49.586927 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:49.585801 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:49.588361 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 350927us
I20260709 12:17:49.552086 21347 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:49.592123 18443 raft_consensus.cc:2410] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:49.574743 21414 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.625298 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:49.628535 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86866.4us
I20260709 12:17:49.629623 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:49.634315 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:49.553072 21522 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:49.632771 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 228506us
W20260709 12:17:49.637702 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 234674us
I20260709 12:17:49.638994 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:49.641067 18338 leader_election.cc:400] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:49.641420 18338 leader_election.cc:403] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.647063 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:49.633214 21497 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.583123 21534 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:17:49.593544 21398 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:17:49.588368 21295 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:17:49.641110 21414 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.640823 21518 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:49.598894 21496 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:17:49.628281 21408 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 12:17:49.659432 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 77912.6us
I20260709 12:17:49.647351 21526 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.634462 21470 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:17:49.631850 21498 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:17:49.627000 21528 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:17:49.678866 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.655247 21374 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.685827 18368 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:49.658041 21538 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.660279 21414 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.688961 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:49.692366 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.693358 18375 raft_consensus.cc:2410] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:49.694988 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.695562 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.675024 21526 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:49.700285 18368 raft_consensus.cc:2376] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6. Re-sending same reply.
I20260709 12:17:49.701812 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.699822 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:49.702723 18367 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 5: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:49.703480 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:49.704375 18368 raft_consensus.cc:2376] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7. Re-sending same reply.
I20260709 12:17:49.709616 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:49.712653 21540 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:49.724802 18447 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:49.728089 21537 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.131s
I20260709 12:17:49.687196 21531 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.692719 21414 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.732501 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:49.695768 21374 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:49.758867 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:49.758985 18446 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:49.763540 18446 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:17:49.763859 18446 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:49.719980 21534 consensus_queue.cc:1243] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:49.731468 21398 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.710299 21526 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.783828 18292 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:17:49.771427 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 115147us
W20260709 12:17:49.778825 21382 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.000s
I20260709 12:17:49.758937 21333 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.781570 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.785492 18297 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:49.781919 20946 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.681385 21518 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.756332 21531 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.788415 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:49.789575 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.788058 18446 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.737776 21532 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:49.794265 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 147256us
I20260709 12:17:49.795720 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.798939 18298 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:49.795327 21382 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.799914 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:49.797080 18291 raft_consensus.cc:2376] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
I20260709 12:17:49.799453 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:49.731827 21520 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:49.691803 21347 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.803313 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.801687 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.815914 21530 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.802688 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:49.807375 18445 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:49.792876 21470 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.812402 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:49.793033 21534 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.816025 18295 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:49.823971 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:49.801211 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:49.824843 18446 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:17:49.803923 21237 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 12:17:49.824908 18300 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:49.832373 19040 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:49.805600 21517 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.806571 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:49.798239 21531 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.793974 21398 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.828676 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:49.884092 19043 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:49.868453 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 238844us
I20260709 12:17:49.890888 19037 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:49.888882 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.813830 21547 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:49.841184 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.899395 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 74889.8us
W20260709 12:17:49.940471 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 147639us
I20260709 12:17:49.892928 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:49.945942 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:49.946664 19038 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.891487 21563 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:49.948696 21546 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:49.948282 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:49.949004 21563 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:49.949119 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 56276.3us
I20260709 12:17:49.949461 21563 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.949621 21556 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.950107 21556 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.950465 21556 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.889631 21382 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:49.891052 21532 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.957137 21414 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.961024 21529 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:17:49.829516 21522 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.961308 21556 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:49.963356 21299 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.000s
I20260709 12:17:49.963734 21299 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:49.964020 20946 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.964010 21556 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.828184 21470 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.965881 21299 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:49.966703 21396 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.000s
I20260709 12:17:49.967154 18373 raft_consensus.cc:2368] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:49.967501 18296 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:49.967797 21470 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.967079 21396 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.842604 21237 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:49.970624 21237 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.850522 21417 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:17:49.972445 21538 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.966465 21299 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:17:49.976003 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.979946 18446 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:17:49.945139 21390 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
W20260709 12:17:49.992380 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44193.5us
I20260709 12:17:49.993367 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:49.829618 21534 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:17:50.003183 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55113.7us
I20260709 12:17:50.003968 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:50.004065 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.004648 19041 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:49.977672 21414 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:49.980557 21396 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:49.987811 21522 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.991729 21374 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:17:50.017220 21543 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:17:50.017409 21374 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.012025 21547 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.020960 21527 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:50.021382 21527 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:50.022269 21527 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.023254 21517 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:50.024101 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 217604us
I20260709 12:17:50.024899 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:50.025693 18412 leader_election.cc:400] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:50.025975 18412 leader_election.cc:403] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:17:50.026873 18339 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.501s (SENT)
I20260709 12:17:50.027360 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:49.979899 21237 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:17:50.027881 21568 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:17:50.018129 21392 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.029109 21563 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.031000 21529 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:50.014344 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:49.998739 21534 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.001814 21299 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.037672 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:50.032001 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.033077 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:50.081197 18446 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:17:50.081578 19042 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:17:50.035522 21241 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.045s	user 0.011s	sys 0.000s
I20260709 12:17:50.082659 19040 raft_consensus.cc:2410] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:50.082381 21241 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.012847 21382 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.080557 21299 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:50.084542 21241 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:50.079857 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53904.7us
W20260709 12:17:50.085685 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 99774us
W20260709 12:17:50.086053 19006 leader_election.cc:336] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.973s (SENT)
I20260709 12:17:50.086428 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:50.088950 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:50.089569 18440 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.089891 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:50.090727 18443 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:50.090795 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 63954.8us
W20260709 12:17:50.091902 18339 leader_election.cc:336] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.653s (SENT)
I20260709 12:17:50.089535 18446 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:17:50.092347 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:50.093124 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.093784 19041 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:50.031191 21563 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:50.096827 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63801.5us
I20260709 12:17:50.094854 21333 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:50.098253 21536 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:17:50.101544 21563 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:50.103479 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 77931.8us
I20260709 12:17:50.103822 21414 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:50.104209 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:50.104219 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 665) took 1117 ms. Trace:
I20260709 12:17:50.104864 21550 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.104905 18447 rpcz_store.cc:276] 0709 12:17:48.987313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.987515 (+   202us) service_pool.cc:224] Handling call
0709 12:17:50.104025 (+1116510us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000d9654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:50.104131 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432343808}
I20260709 12:17:50.106241 21532 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:50.106654 18447 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:17:50.106743 21537 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.107244 21537 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.107570 21537 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.108655 21518 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.109488 18369 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:50.109846 18369 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:50.035112 21542 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.109810 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:50.085592 21573 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:50.110742 18369 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.112900 19038 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:17:50.120599 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 95860us
I20260709 12:17:50.125583 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:50.126178 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:50.127377 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:50.087742 21538 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:17:50.139482 21538 consensus_queue.cc:1243] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:50.140828 19037 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:50.120680 21392 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:50.122910 21398 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.143635 21531 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.146066 21531 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:50.148937 21509 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.835s	user 0.003s	sys 0.008s
I20260709 12:17:50.130616 21532 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:50.131500 21504 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:50.132617 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:50.166885 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:50.133152 21518 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:50.113289 21542 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:50.135099 21382 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:50.135455 21564 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:50.138054 21534 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:50.150319 18375 raft_consensus.cc:2368] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:50.150302 18369 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:50.151487 18339 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.597s (SENT)
I20260709 12:17:50.201280 18440 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:17:50.201900 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50431.6us
I20260709 12:17:50.084846 21299 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:50.202726 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116700us
I20260709 12:17:50.032081 21522 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.168989 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:50.210505 18447 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:50.119468 21241 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.198182 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.204602 19037 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:17:50.212433 18300 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:17:50.215070 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 220049us
I20260709 12:17:50.211819 21237 raft_consensus.cc:993] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:17:50.215718 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:50.149452 21509 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.199824 21573 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:50.215689 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:50.200191 21504 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:50.217626 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:50.221583 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.223387 21299 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.210295 21398 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.210114 21564 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:50.229470 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:50.201157 21382 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.231774 21577 raft_consensus.cc:1081] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:17:50.231879 18369 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:17:50.232326 21496 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.232352 18292 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:50.233495 21578 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.233565 21534 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.225682 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:50.235457 18444 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:50.242643 18369 pending_rounds.cc:85] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:17:50.234440 21573 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.224628 21538 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:17:50.247315 21581 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:50.247545 21504 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.248616 21382 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:50.249295 21557 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:50.253692 18369 pending_rounds.cc:107] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:17:50.266616 18369 raft_consensus.cc:2889] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:50.267469 21542 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.225660 21398 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.255664 18441 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:50.259603 18291 raft_consensus.cc:1217] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Deduplicated request from leader. Original: 2.5->[4.6-4.6]   Dedup: 4.6->[]
I20260709 12:17:50.268900 21398 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.269217 18291 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:50.262998 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.263688 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.271150 19041 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
I20260709 12:17:50.265357 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.276201 18297 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:50.280560 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.285379 19040 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:50.288687 18293 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:17:50.412065 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.440405 18292 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:50.440650 21509 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:50.449568 18440 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:50.471087 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.455189 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 229555us
I20260709 12:17:50.479907 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.480284 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:50.289366 21407 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:50.281533 21581 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:50.486989 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.266288 21557 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:50.450743 21542 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.497807 18296 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:50.442085 21470 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:50.480782 21560 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:50.496958 21347 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:50.524034 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 43813.8us
I20260709 12:17:50.526295 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:50.528683 21555 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:17:50.252599 21564 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.533339 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:50.529863 21537 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.545071 21398 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.532099 21347 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.560163 21330 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.527179 21396 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:50.542133 21578 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:50.535693 21522 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.521263 21407 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:50.657131 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:50.671371 18300 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:50.659339 21518 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:50.700227 21586 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
W20260709 12:17:50.688788 18443 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.591s	user 0.006s	sys 0.014s
I20260709 12:17:50.544811 21581 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.691008 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:50.725986 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68948.4us
I20260709 12:17:50.697497 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46944 (request call id 20) took 1340 ms. Trace:
I20260709 12:17:50.656286 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.699672 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.511859 21470 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:50.658170 21585 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.137s	user 0.001s	sys 0.001s
I20260709 12:17:50.703593 18446 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:50.749107 18296 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.749727 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:50.750239 18297 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.750540 18369 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:50.750715 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:50.752408 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:50.753010 18369 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.754382 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.754784 21580 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.755416 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.755323 21580 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.755824 21347 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:50.756017 18291 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.757579 21580 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:50.758337 21299 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.000s
I20260709 12:17:50.744729 21377 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:50.759953 21563 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.760437 21563 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:50.760962 21592 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:17:50.761119 21470 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.761315 21563 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.762526 21563 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.762627 21377 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:50.722529 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.763070 21377 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.763188 18295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:45180
I20260709 12:17:50.763518 21522 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:17:50.764000 18295 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.764334 18295 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.764567 21522 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:50.764926 21522 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:50.765753 21522 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.766779 21518 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.769188 21518 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.721096 21578 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.771623 21578 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.772320 21347 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.772732 21497 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.772817 21347 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:50.698846 21526 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.002s
I20260709 12:17:50.723783 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.773909 21526 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.775687 21527 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.775997 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:50.699344 21330 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.775730 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.776782 21330 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.776969 21230 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:50.776057 21526 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.777549 21230 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:50.759874 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:50.778689 18371 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:50.773301 21497 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:50.778568 21230 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.779953 21497 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.780965 21497 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.723269 21557 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.781910 18368 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:50.782159 21557 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.776046 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.783888 19042 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:50.721020 21396 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:50.785848 21396 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.787372 21517 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.787796 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.787850 21517 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.788983 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.790133 21517 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.790627 21517 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.727398 21407 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.793321 21407 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:50.775032 21347 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.791013 21517 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:50.498593 21562 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.267s
I20260709 12:17:50.726998 18448 rpcz_store.cc:276] 0709 12:17:49.347141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.347282 (+   141us) service_pool.cc:224] Handling call
0709 12:17:50.551143 (+1203861us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54000d8c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e82735 00007f3c6de42d29 00007f3c673c11d4 00007f3c673c1162 00007f3c673c1091 00007f3c673c100c 00007f3c673c02c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:50.687298 (+136155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978293504}
I20260709 12:17:50.725229 18443 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:50.743712 21537 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.801534 18300 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.802037 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1765) took 1210 ms. Trace:
I20260709 12:17:50.839601 18445 rpcz_store.cc:276] 0709 12:17:49.591904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.623642 (+ 31738us) service_pool.cc:224] Handling call
0709 12:17:50.801855 (+1178213us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:50.801985 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384108416}
I20260709 12:17:50.843614 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:50.665373 18412 connection.cc:256] RPC call timeout handler was delayed by -3.5805951799998184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 12:17:50.795794 21566 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:17:50.853721 18300 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:50.852677 18296 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:50.815533 18291 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:50.836606 18369 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:50.861156 18297 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:50.861887 18443 raft_consensus.cc:1217] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 3.5->[6.6-10.8]   Dedup: 6.6->[8.7-10.8]
I20260709 12:17:50.795516 18295 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.885753 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.886677 18300 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:50.886737 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.887599 21535 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:50.887750 18296 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.888156 21535 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:50.888689 18295 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:50.889626 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.890205 18295 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:50.890295 21535 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.891158 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:50.892006 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 7d788f8182ca4b13bddd2acdf511873f: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:50.892431 18295 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.863268 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:50.894464 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:50.895732 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.886157 18443 pending_rounds.cc:85] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:17:50.798007 21517 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.898299 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.924840 18291 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:50.925767 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.926364 18291 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:50.827387 21522 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.904898 18443 pending_rounds.cc:107] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 9.7
W20260709 12:17:50.907541 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 45389.8us
I20260709 12:17:50.935818 18443 raft_consensus.cc:2889] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:50.919286 18296 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:50.852959 21504 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:50.899852 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 364615us
I20260709 12:17:50.927636 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:50.943328 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.602s (SENT)
W20260709 12:17:50.943369 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.927938 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:50.927219 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.945004 18291 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:50.930428 18415 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.018s	sys 0.017s
W20260709 12:17:50.946590 18415 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.018s	sys 0.018s
I20260709 12:17:50.794631 21330 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:50.946880 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 612154us
I20260709 12:17:50.828822 21398 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:50.967324 18412 leader_election.cc:336] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.605s (SENT)
I20260709 12:17:50.967939 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:50.969163 18412 connection.cc:256] RPC call timeout handler was delayed by -3.5193951799999468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
W20260709 12:17:50.969857 18412 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.706s (SENT)
W20260709 12:17:50.970820 18412 connection.cc:256] RPC call timeout handler was delayed by -3.3258951799998613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 12:17:50.972590 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.975872 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.976338 18412 connection.cc:256] RPC call timeout handler was delayed by -2.8149179629999708s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 12:17:50.976806 18412 connection.cc:256] RPC call timeout handler was delayed by -2.5962179630000719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 12:17:50.977178 18412 connection.cc:256] RPC call timeout handler was delayed by -2.5671951799997714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3027s for a response
W20260709 12:17:50.977555 18412 connection.cc:256] RPC call timeout handler was delayed by -2.4396179630000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 12:17:50.977926 18412 connection.cc:256] RPC call timeout handler was delayed by -2.279365407999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 12:17:50.942080 18369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1f155085cb647ff957b468384227872"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
W20260709 12:17:50.978292 18412 connection.cc:256] RPC call timeout handler was delayed by -2.2443179629999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 12:17:50.978699 18412 connection.cc:256] RPC call timeout handler was delayed by -2.2217654079997828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 12:17:50.978725 18369 raft_consensus.cc:606] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:50.979139 18412 connection.cc:256] RPC call timeout handler was delayed by -2.1911654079999607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
I20260709 12:17:50.979398 18369 raft_consensus.cc:614] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:50.979645 18412 connection.cc:256] RPC call timeout handler was delayed by -2.1794654079999418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 12:17:50.927778 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:50.980078 18412 connection.cc:256] RPC call timeout handler was delayed by -2.0615654079999786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
W20260709 12:17:50.980768 18412 connection.cc:256] RPC call timeout handler was delayed by -1.9238654079997559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 12:17:50.980937 18300 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:50.948835 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:50.982088 18368 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.942999 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:50.950822 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.951442 18366 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:17:50.983219 18442 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:50.953392 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.953738 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:50.956136 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:50.958482 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.923898 21568 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:17:51.013226 19043 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:50.959055 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:50.959422 21538 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:50.960090 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:50.962035 21330 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:50.953006 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:50.981190 18412 connection.cc:256] RPC call timeout handler was delayed by -1.8977654079999411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:17:51.026854 18412 connection.cc:256] RPC call timeout handler was delayed by -1.8869654079999236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 12:17:51.017716 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:51.029635 18440 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:50.949400 21295 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:51.014220 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:51.015466 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:51.036361 18300 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:51.018630 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:50.834178 21540 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.181s
I20260709 12:17:51.037285 18296 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:51.019217 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.014864 18295 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:50.834311 21555 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.141s
I20260709 12:17:50.940276 21535 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:51.054380 21535 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:51.054889 21535 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.044435 21398 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:51.020722 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:51.057461 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 309421us
I20260709 12:17:51.025182 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.058946 19041 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:51.059204 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:51.060047 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.019280 21537 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.984146 18291 raft_consensus.cc:2376] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7. Re-sending same reply.
I20260709 12:17:51.034979 18448 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:51.066104 18448 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:51.037149 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:51.067306 18300 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:51.016760 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:51.039124 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.048929 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:51.049811 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.068141 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51500.6us
I20260709 12:17:51.050136 19035 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:51.050570 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.052153 18295 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:51.073457 18295 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:17:51.074635 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.053751 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:51.030577 21538 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:51.064903 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:51.031075 21578 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.031352 21526 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:51.027355 21504 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:51.068837 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:51.035554 21568 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.036008 21295 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:51.032797 21377 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:51.308684 19043 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.120270 18448 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.135510 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:51.431829 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 366972us
I20260709 12:17:51.181919 18440 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:51.180759 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 524) took 1377 ms. Trace:
I20260709 12:17:51.119556 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:51.197086 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 149467us
I20260709 12:17:51.134158 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:51.160269 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:51.090479 18296 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:51.435468 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 315979us
I20260709 12:17:51.310623 18293 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
W20260709 12:17:51.435194 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 398367us
I20260709 12:17:51.434361 18294 rpcz_store.cc:276] 0709 12:17:49.790567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.790702 (+   135us) service_pool.cc:224] Handling call
0709 12:17:49.799354 (+  8652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.799673 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.799718 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.800154 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:17:50.542875 (+742721us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b18003648a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1
0709 12:17:50.571132 (+ 28257us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:50.696490 (+125358us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.790742 (+ 94252us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18003648a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:17:51.056689 (+265947us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b18003648a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:51.167468 (+110779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.167575 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.167636 (+    61us) raft_consensus.cc:1685] finished
0709 12:17:51.167655 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.167833 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124909,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1343,"raft.queue_time_us":107096,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":35236,"spinlock_wait_cycles":138826624,"thread_start_us":492368,"threads_started":2,"wal-append.queue_time_us":221960}
I20260709 12:17:51.439463 18266 leader_election.cc:400] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:51.443882 18266 leader_election.cc:403] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:51.443296 21295 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:51.445968 21295 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:51.146193 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:51.205763 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.449990 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:51.451763 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.452980 18370 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:51.196384 18373 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.134976 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:51.589057 18445 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:51.454041 18295 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:17:51.483084 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:51.589998 18295 pending_rounds.cc:85] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 6
I20260709 12:17:51.590396 18295 pending_rounds.cc:107] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:17:51.590672 18295 raft_consensus.cc:2889] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:51.590708 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 107696us
I20260709 12:17:51.453794 21538 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.338452 19035 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.091384 21407 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:51.149044 21499 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.598785 21499 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:51.489249 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 420452us
W20260709 12:17:51.492619 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53332.7us
I20260709 12:17:51.493048 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 593) took 1271 ms. Trace:
I20260709 12:17:51.492712 21568 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:51.493932 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.493813 21377 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:51.079886 21508 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.410s
I20260709 12:17:51.495056 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.487114 21347 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.496119 21560 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
W20260709 12:17:51.496858 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.495049 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.630412 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.631192 18371 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:51.632225 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.461361 21573 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:51.460539 21542 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.510376 21504 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.513875 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.485204 21520 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.503171 21527 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.460281 21526 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.460101 21578 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:51.520738 18368 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.001s
I20260709 12:17:51.488915 21550 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:51.314150 19036 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:17:51.539978 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.553380 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:51.591112 18295 pending_rounds.cc:107] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:17:51.072322 21398 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:51.671221 21604 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:17:51.591833 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:51.596935 18298 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:51.672405 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 80681.8us
I20260709 12:17:51.672602 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1713) took 1422 ms. Trace:
W20260709 12:17:51.600739 18375 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1798) took 1901 ms (client timeout 1878 ms). Trace:
I20260709 12:17:51.600950 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:51.672982 21531 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:51.673471 21573 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.673959 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:51.674109 21573 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.674489 18372 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:51.674865 18372 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:51.601346 18446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 679) took 1476 ms (client timeout 1727 ms). Trace:
I20260709 12:17:51.607606 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.675079 21573 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:51.675707 18293 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:51.676234 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:51.605017 21597 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:51.676862 18367 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:51.676996 21597 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:51.608786 18443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.069s	sys 0.113s
W20260709 12:17:51.678458 18443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.069s	sys 0.113s
I20260709 12:17:51.636904 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.679756 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:51.680938 21535 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.626s	user 0.004s	sys 0.006s
I20260709 12:17:51.681005 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:51.681473 21499 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:51.681653 18371 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.682397 21382 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:51.594708 21407 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:51.652011 18296 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.651325 21520 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.684825 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:51.653649 18368 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:51.685341 21520 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:51.655628 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:51.685817 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.686455 18292 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:17:51.686515 18368 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:51.687347 18368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:17:51.687809 18368 raft_consensus.cc:606] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.648272 21347 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.688119 18368 raft_consensus.cc:614] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:51.689204 18368 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
W20260709 12:17:51.689600 21522 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.704s	user 0.008s	sys 0.000s
I20260709 12:17:51.689692 21347 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:51.690714 21497 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.909s	user 0.010s	sys 0.002s
I20260709 12:17:51.667383 18295 raft_consensus.cc:2889] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:51.675364 18446 rpcz_store.cc:269] 0709 12:17:50.125071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.125259 (+   188us) service_pool.cc:224] Handling call
0709 12:17:51.601168 (+1475909us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54000d8254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:51.601288 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327152000}
I20260709 12:17:51.676364 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.605643 19038 rpcz_store.cc:276] 0709 12:17:50.221233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.221355 (+   122us) service_pool.cc:224] Handling call
0709 12:17:51.492826 (+1271471us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54005316d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:51.492963 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149972864}
I20260709 12:17:51.679695 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.682653 21526 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:51.683441 21407 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.606257 21563 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.685405 19037 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.670499 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.675552 18372 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.706336 19043 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
W20260709 12:17:51.670543 21582 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.421s	user 0.006s	sys 0.000s
I20260709 12:17:51.681457 21535 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:51.685345 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.714114 18366 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:51.715476 18366 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:51.691092 18368 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:51.672993 18298 rpcz_store.cc:276] 0709 12:17:50.249816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.257647 (+  7831us) service_pool.cc:224] Handling call
0709 12:17:51.596578 (+1338931us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b5800003698. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57f8c 00007f3c65d57ef6 000000000036eb39 00007f3c6ddcb1e0 00007f3c6de940c4 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:17:51.596751 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.672362 (+ 75611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.672550 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4400128}
I20260709 12:17:51.676930 18293 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:51.673094 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:51.697067 18370 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:51.700902 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:51.717401 18446 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.701699 18297 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:51.706425 18372 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:51.673130 18375 rpcz_store.cc:269] 0709 12:17:49.699792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.700533 (+   741us) service_pool.cc:224] Handling call
0709 12:17:51.600506 (+1899973us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002e7a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:51.600654 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029384576}
I20260709 12:17:51.651677 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:51.707525 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 8903d70080d24236be955433a92791c7: Illegal state: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:51.708954 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:51.673520 21531 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.710484 18371 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:51.711621 18374 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:17:51.746582 18294 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.754400 19038 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:51.754959 18446 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
W20260709 12:17:51.754747 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 300027us
I20260709 12:17:51.755978 18445 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 12:17:51.756995 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 83980.2us
W20260709 12:17:51.757222 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48352.2us
I20260709 12:17:51.757678 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:51.757915 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.764129 18366 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:17:51.767789 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.768031 19037 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:51.769352 18375 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:51.777489 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.778234 18372 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:51.669277 21550 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:51.786084 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 272266us
I20260709 12:17:51.792148 18445 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:51.747354 21563 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.796625 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:51.798213 18372 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:51.798373 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 791) took 1081 ms. Trace:
I20260709 12:17:51.807672 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:51.808880 18446 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:51.764781 21499 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.773649 21241 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.789186 21520 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.818055 21597 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:51.817922 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.818599 21396 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:17:51.819291 21347 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:51.818997 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:51.793452 21564 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:51.823596 18372 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:51.819017 21599 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
I20260709 12:17:51.809490 21526 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.807924 21497 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:51.803949 19040 rpcz_store.cc:276] 0709 12:17:50.704601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.704734 (+   133us) service_pool.cc:224] Handling call
0709 12:17:50.705138 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:50.716127 (+ 10989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:50.716147 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:50.716492 (+   345us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.190913 (+474421us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:51.778371 (+587458us) log.cc:844] Serialized 14 byte log entry
0709 12:17:51.785875 (+  7504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.785974 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.786031 (+    57us) raft_consensus.cc:1685] finished
0709 12:17:51.786045 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.786191 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7146,"prepare.queue_time_us":353863,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":42190,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":12334,"thread_start_us":791215,"threads_started":2,"wal-append.queue_time_us":838180}
I20260709 12:17:51.878525 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.826308 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.886628 18367 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:17:51.903019 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:51.879902 18294 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:51.915707 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119164us
I20260709 12:17:51.888513 21622 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:51.879601 19036 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:17:51.869621 21508 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:51.892058 21620 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:17:51.935846 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 158438us
I20260709 12:17:51.936596 18372 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.936870 19043 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:51.868264 21564 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:51.869621 21520 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.953173 18372 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:17:51.957516 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.967309 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.821836 21531 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:51.919840 21526 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:51.928722 21622 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:51.955327 21564 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.982443 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.988517 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:51.955288 21623 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:51.995184 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1686) took 1245 ms. Trace:
I20260709 12:17:51.995299 18372 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:17:51.993731 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 115291us
I20260709 12:17:51.995667 18373 rpcz_store.cc:276] 0709 12:17:50.749860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.751215 (+  1355us) service_pool.cc:224] Handling call
0709 12:17:50.767749 (+ 16534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.591622 (+823873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.994974 (+403352us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:51.995114 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985163648}
I20260709 12:17:51.996897 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:51.998055 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.79537e+06us
W20260709 12:17:51.998939 18339 leader_election.cc:336] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.679s (SENT)
I20260709 12:17:51.999361 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:51.964174 21597 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 12:17:51.992677 21606 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 12:17:51.965631 21598 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:17:51.963393 21508 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.001258 21573 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:51.997449 21531 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.009073 21626 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.990413 21578 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.027338 21535 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:52.016150 21613 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:17:52.031544 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 535) took 1217 ms. Trace:
W20260709 12:17:52.032471 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.031914 18300 rpcz_store.cc:276] 0709 12:17:50.813841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.072920 (+259079us) service_pool.cc:224] Handling call
0709 12:17:51.073193 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:51.073381 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:51.073404 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:51.080910 (+  7506us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.637059 (+556149us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:51.637109 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.637171 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:52.031342 (+394171us) raft_consensus.cc:1685] finished
0709 12:17:52.031361 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:52.031485 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":318594,"threads_started":1,"wal-append.queue_time_us":613753}
I20260709 12:17:52.036192 21520 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.039645 21550 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:52.050535 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 444736us
W20260709 12:17:52.050935 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.056531 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.004359 18339 connection.cc:256] RPC call timeout handler was delayed by -1.8127263410001433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 12:17:52.011008 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1769) took 1973 ms. Trace:
W20260709 12:17:52.058763 18339 connection.cc:256] RPC call timeout handler was delayed by -1.7434551240000928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260709 12:17:52.058802 18441 rpcz_store.cc:276] 0709 12:17:50.037857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.085640 (+ 47783us) service_pool.cc:224] Handling call
0709 12:17:50.093113 (+  7473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:50.268969 (+175856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.010773 (+1741804us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000d8c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:52.010947 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373400832}
I20260709 12:17:51.994251 21542 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:52.010324 21526 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:17:52.016124 21610 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:17:52.047894 21538 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.004426 21612 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:17:52.017880 21622 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.018826 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.023046 19035 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.007431 21497 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:17:51.965907 21563 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.060491 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 86330.2us
I20260709 12:17:52.063551 21623 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.101467 19044 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:52.078019 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.079273 18300 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:17:52.059353 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.921s (SENT) [suppressed 63 similar messages]
W20260709 12:17:52.103020 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43692.7us
I20260709 12:17:52.103217 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.082304 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:52.087604 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:52.080229 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 163396us
I20260709 12:17:52.082175 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:52.063270 18294 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:52.103603 18339 leader_election.cc:336] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.537s (SENT)
I20260709 12:17:52.108522 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.017225 21241 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:52.112216 21241 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:52.112962 21555 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.113502 21555 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.113651 21241 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.112269 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 93543.1us
I20260709 12:17:52.116681 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.117937 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.118850 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:52.119971 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.083993 21635 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:52.091452 21557 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.081717 21578 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:17:52.081281 21616 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:17:52.094928 21527 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:52.145402 18294 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:52.094666 21626 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:52.100298 21398 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:17:52.143561 21602 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.119374 21540 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:52.161792 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.159838 21623 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.125172 21241 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:52.123572 21528 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.103827 21573 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.080947 21625 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:17:52.148404 21517 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:52.170213 21610 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:17:52.162822 21640 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.094201 21377 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:17:52.164587 21628 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.244525 21597 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.257791 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 651791us
I20260709 12:17:52.210187 21598 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.210855 21537 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.067s
W20260709 12:17:52.261278 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 99606.4us
I20260709 12:17:52.167361 21631 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 12:17:52.180164 21557 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.389318 21557 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.390786 21522 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.391377 21522 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.393535 21522 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.394198 21522 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:17:52.247190 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 129371us
I20260709 12:17:52.275837 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:52.259684 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 533) took 1446 ms. Trace:
I20260709 12:17:52.199532 21527 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.207738 21407 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.399464 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.264787 21517 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.308076 21629 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:17:52.308403 21538 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:52.278445 21597 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.258461 21528 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.311594 21333 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.311700 21508 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.305498 21598 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.263656 21555 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.273559 21628 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.315963 21633 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:17:52.245944 21602 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.316118 21564 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.212255 21542 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:52.319326 21647 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:17:52.319748 21656 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:52.320452 21626 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:52.321108 21607 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.653s	user 0.002s	sys 0.000s
I20260709 12:17:52.323432 21622 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.327409 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 681) took 1481 ms. Trace:
I20260709 12:17:52.447811 21557 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:52.326843 21640 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.340716 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 601) took 1320 ms. Trace:
I20260709 12:17:52.465790 18443 rpcz_store.cc:276] 0709 12:17:50.846188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.846324 (+   136us) service_pool.cc:224] Handling call
0709 12:17:50.846806 (+   482us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:50.941235 (+ 94429us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:50.964789 (+ 23554us) log.cc:844] Serialized 14 byte log entry
0709 12:17:51.847654 (+882865us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:51.866091 (+ 18437us) log.cc:844] Serialized 52 byte log entry
0709 12:17:51.866292 (+   201us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:51.866342 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.866432 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:52.327170 (+460738us) raft_consensus.cc:1685] finished
0709 12:17:52.327188 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:52.327345 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":969595,"prepare.queue_time_us":36628,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":5111,"raft.queue_time_us":316234,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":18405,"thread_start_us":882394,"threads_started":2,"wal-append.queue_time_us":851574}
I20260709 12:17:52.467864 19042 rpcz_store.cc:276] 0709 12:17:51.019968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.038846 (+ 18878us) service_pool.cc:224] Handling call
0709 12:17:52.340471 (+1301625us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540066d854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:52.340645 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432996224}
W20260709 12:17:52.341939 21618 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.671s	user 0.002s	sys 0.000s
I20260709 12:17:52.342598 19041 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:52.469405 21564 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:52.306082 21535 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:52.471339 21535 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.351356 21637 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 12:17:52.353334 21623 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.367272 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1719) took 1572 ms. Trace:
I20260709 12:17:52.376955 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 794) took 1419 ms. Trace:
W20260709 12:17:52.385433 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104621us
W20260709 12:17:52.476301 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 356395us
I20260709 12:17:52.452137 21628 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:52.265448 21295 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.006s
I20260709 12:17:52.398180 18291 rpcz_store.cc:276] 0709 12:17:50.813315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.070028 (+256713us) service_pool.cc:224] Handling call
0709 12:17:51.070640 (+   612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:51.071040 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:51.071080 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:51.083323 (+ 12243us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.576069 (+492746us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:51.584220 (+  8151us) log.cc:844] Serialized 14 byte log entry
0709 12:17:52.050239 (+466019us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b18004114c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:17:52.050478 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.050576 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:52.252219 (+201643us) raft_consensus.cc:1685] finished
0709 12:17:52.259393 (+  7174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:52.259607 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55629,"prepare.queue_time_us":4200,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":475718,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":37541,"spinlock_wait_cycles":3771264,"thread_start_us":394640,"threads_started":1,"wal-append.queue_time_us":520562}
I20260709 12:17:52.399467 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.434216 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:52.480472 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 616) took 1084 ms. Trace:
W20260709 12:17:52.480545 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 81152.5us
W20260709 12:17:52.436100 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 160469us
I20260709 12:17:52.443873 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:52.449308 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.402554 21517 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.436764 21382 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:17:52.345589 18448 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.386031 19039 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:52.483671 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 406756us
I20260709 12:17:52.477072 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.491456 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.476171 18297 rpcz_store.cc:276] 0709 12:17:50.786422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.927399 (+140977us) service_pool.cc:224] Handling call
0709 12:17:52.358871 (+1431472us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54002760d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:52.359006 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716643712}
W20260709 12:17:52.478139 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 358916us
I20260709 12:17:52.476491 19044 rpcz_store.cc:276] 0709 12:17:50.958022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.958165 (+   143us) service_pool.cc:224] Handling call
0709 12:17:52.376719 (+1418554us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400545bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:52.376890 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502332800}
I20260709 12:17:52.482510 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.488204 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.482574 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.494861 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.481000 18369 rpcz_store.cc:276] 0709 12:17:51.396031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.448234 (+ 52203us) service_pool.cc:224] Handling call
0709 12:17:52.433989 (+985755us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5800083a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:17:52.480407 (+ 46418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25256704}
I20260709 12:17:52.496810 18445 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:17:52.477965 21623 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:52.499647 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.503917 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 853) took 1452 ms. Trace:
I20260709 12:17:52.482074 21557 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:52.485783 21640 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:52.508190 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 402633us
W20260709 12:17:52.510452 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.506479 19041 rpcz_store.cc:276] 0709 12:17:51.051432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.147617 (+ 96185us) service_pool.cc:224] Handling call
0709 12:17:52.342277 (+1194660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.469712 (+127435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.503768 (+ 34056us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540066d850. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:52.503871 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2634176768}
I20260709 12:17:52.487309 21538 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.512537 21598 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.513463 21578 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:52.521394 21555 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.513993 21540 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:52.521672 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.522063 21555 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.522498 21540 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.495604 21535 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.519126 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.499647 21295 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:52.521463 21578 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:17:52.518155 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.520548 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.531635 18444 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:52.531708 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 852) took 1516 ms. Trace:
I20260709 12:17:52.532232 19039 rpcz_store.cc:276] 0709 12:17:51.015203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.052607 (+ 37404us) service_pool.cc:224] Handling call
0709 12:17:52.380111 (+1327504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.488451 (+108340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.531512 (+ 43061us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400545bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:52.531652 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3052138240}
I20260709 12:17:52.496322 21398 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:52.517969 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.535269 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.523926 21616 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:17:52.525734 18442 raft_consensus.cc:2368] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9.
I20260709 12:17:52.505587 21634 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:52.444077 21653 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:17:52.463728 21656 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.472771 21565 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:17:52.514741 21640 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.540235 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48868.1us
I20260709 12:17:52.527511 21295 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.542876 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65888.9us
I20260709 12:17:52.543799 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.544701 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.547106 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.547046 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:17:52.555087 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 451467us
I20260709 12:17:52.524099 21577 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.504738 21382 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:52.542565 18448 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:17:52.514019 21655 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.553153 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.565186 18448 pending_rounds.cc:85] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 7
W20260709 12:17:52.555616 18339 connection.cc:256] RPC call timeout handler was delayed by -1.6628822280001714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
I20260709 12:17:52.555925 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.565485 18448 pending_rounds.cc:107] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:17:52.507619 21508 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:52.565766 18448 raft_consensus.cc:2889] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:52.565790 18339 connection.cc:256] RPC call timeout handler was delayed by -1.6028470769999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 12:17:52.566109 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.566524 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 690) took 1550 ms. Trace:
W20260709 12:17:52.566480 18339 connection.cc:256] RPC call timeout handler was delayed by -1.5731470769999305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 12:17:52.567276 18339 connection.cc:256] RPC call timeout handler was delayed by -1.5360640549999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
I20260709 12:17:52.567072 18448 rpcz_store.cc:276] 0709 12:17:51.015845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.016043 (+   198us) service_pool.cc:224] Handling call
0709 12:17:51.016380 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.345430 (+1329050us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b580063b0d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68dd80aa 00007f3c68e5a447 00007f3c68e72fe7 00007f3c68e747de 00007f3c68e7eb49 00007f3c68e7f416 00007f3c68e7bd8f 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4
0709 12:17:52.566315 (+220885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.566479 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677883136}
I20260709 12:17:52.569242 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.570664 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.571581 18444 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:52.571866 18448 raft_consensus.cc:2368] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
I20260709 12:17:52.572856 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.573076 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.574051 18444 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:17:52.574928 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.575244 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.576864 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.578385 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.580897 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.582266 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.582947 18443 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:52.586536 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:52.588629 18339 connection.cc:256] RPC call timeout handler was delayed by -1.4251403289999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 12:17:52.589336 18339 connection.cc:256] RPC call timeout handler was delayed by -1.2817291009998826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 12:17:52.589762 18339 connection.cc:256] RPC call timeout handler was delayed by -1.2465688550000777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260709 12:17:52.548800 21555 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.543941 21602 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:52.655493 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.616483 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.673128 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:52.674199 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:52.664534 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.675559 18447 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:52.664389 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:52.665118 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 150106us
I20260709 12:17:52.554105 21640 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.666282 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:52.587348 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 87964.7us
I20260709 12:17:52.669517 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.681511 18367 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:52.681638 18440 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.564640 21550 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.549419 21660 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.579533 21534 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.683451 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 127590us
I20260709 12:17:52.697088 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.585171 21531 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:17:52.564841 21295 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:52.707647 21563 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:52.708956 21563 consensus_queue.cc:1243] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:52.714162 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.714212 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:52.715250 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet e9decd1c7d7f475b8890a8519847dca5: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:52.663379 21526 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:52.718209 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49101.6us
W20260709 12:17:52.714246 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 121946us
I20260709 12:17:52.666524 21653 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:52.742508 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.744172 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:52.745431 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48481.1us
I20260709 12:17:52.746407 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:52.663245 21520 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:52.671348 21407 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.759460 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.670945 21527 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.681143 21528 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:52.709512 21563 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:17:52.789206 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.791461 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.792886 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.793640 18443 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
W20260709 12:17:52.760623 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.775471 18444 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.774854 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:52.583359 21577 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.775620 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.785143 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.591135 21657 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.164s
I20260709 12:17:52.720043 21655 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:52.666388 21347 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:17:52.718729 21597 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:52.786209 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 199757us
I20260709 12:17:52.786708 21377 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:52.798672 18440 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:52.671200 21578 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.769343 21602 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:52.849943 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64885.9us
I20260709 12:17:52.851625 18339 leader_election.cc:400] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 5 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6.
W20260709 12:17:52.820326 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1561462209999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:17:52.853942 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1498462209999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 12:17:52.854406 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1336462209999354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 12:17:52.854822 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0202462209999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
W20260709 12:17:52.855247 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0139462209999692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
I20260709 12:17:52.776953 21550 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:52.824084 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.781687 21598 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.825452 18368 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:52.782401 21568 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.796555 18447 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.865013 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:52.865653 19041 raft_consensus.cc:2376] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6. Re-sending same reply.
W20260709 12:17:52.829149 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 153730us
I20260709 12:17:52.722613 21534 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:52.789500 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:52.870234 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:52.848871 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.857524 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.848150 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.871122 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.871428 19037 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:52.859716 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.741710 21644 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:17:52.685176 21557 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.823850 21528 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:52.872696 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.873214 18440 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:52.865887 21634 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:52.869714 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.883303 19035 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:52.802778 21526 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:17:52.823872 21652 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.878013 19039 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:52.878538 19042 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:52.858646 21398 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.802623 21581 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:17:52.830667 21377 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.865034 21568 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:52.884444 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:52.846212 21655 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.787865 21520 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.894850 21654 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.878985 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.893509 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:52.894603 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:52.895503 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:52.905001 18374 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
I20260709 12:17:52.866537 21555 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:52.899144 19041 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:17:52.897243 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 150930us
I20260709 12:17:52.827020 21560 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:17:52.910612 18445 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:52.890487 21526 consensus_queue.cc:1243] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:52.916436 18444 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:52.890676 21652 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:52.927579 18447 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:52.865074 21602 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:52.930507 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 73076.1us
I20260709 12:17:52.895179 21631 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.899914 21377 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.933401 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.894995 21673 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:52.901310 21550 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:52.869495 21598 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.915920 21564 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:52.943615 21669 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.907543 21555 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:52.955653 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.931751 21656 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.898836 21398 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:17:52.959532 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 80642.5us
I20260709 12:17:52.964547 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.937662 21652 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.972404 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
I20260709 12:17:52.976366 18412 leader_election.cc:403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:17:52.951491 21550 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.915364 21499 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:17:52.984388 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.987500 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:52.987679 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:52.988324 18447 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:52.954272 21528 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:53.019202 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:53.020327 18447 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:52.977985 21564 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:52.907306 21655 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.019320 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:52.983723 21634 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.978482 21527 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.978252 21669 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:53.000496 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:52.990913 21550 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.947052 21631 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:52.947836 21642 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.255s
I20260709 12:17:52.998035 21581 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:52.995105 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84517.7us
I20260709 12:17:52.995963 21662 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.029431 21382 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.029934 21526 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:53.054272 21527 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.050988 21577 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:52.972482 21568 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.043831 21675 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.028340 21664 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:53.076038 21688 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
W20260709 12:17:53.063874 21658 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
W20260709 12:17:53.065641 21661 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.372s	user 0.001s	sys 0.001s
I20260709 12:17:53.061442 21672 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.038888 21555 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.068379 21642 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:53.066275 21534 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:53.075425 21527 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.067646 21526 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:53.086512 21577 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:53.109676 21681 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:17:53.145622 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 457072us
I20260709 12:17:53.164062 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:53.164527 19040 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:17:53.164713 18297 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:53.109859 21520 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.067116 21528 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.170934 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:53.171173 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:53.173839 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:53.112452 21526 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.172034 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 700) took 1044 ms. Trace:
I20260709 12:17:53.175153 21678 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:53.175835 21678 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:53.176738 21678 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.174616 18446 rpcz_store.cc:276] 0709 12:17:52.127651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.145910 (+ 18259us) service_pool.cc:224] Handling call
0709 12:17:52.146292 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.171631 (+ 25339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.171675 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:52.172326 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:17:52.451552 (+279226us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:17:52.451579 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.451624 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.171784 (+720160us) raft_consensus.cc:1685] finished
0709 12:17:53.171812 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.171963 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24621,"raft.queue_time_us":96457,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":14866,"thread_start_us":198429,"threads_started":1,"wal-append.queue_time_us":289032}
I20260709 12:17:53.178336 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:53.100222 21671 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:53.109704 21598 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:53.144904 21673 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:17:53.180390 21692 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:17:53.180647 21673 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:53.179308 21665 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 12:17:53.181515 21691 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:53.068677 21662 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:53.207314 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.209838 21623 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.112164 21564 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.161013 21699 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 12:17:53.337057 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.406277 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.522006 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:53.516911 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:53.542552 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:53.512109 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 511656us
W20260709 12:17:53.489182 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 276618us
I20260709 12:17:53.331439 21664 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.554909 21664 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:53.161059 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 188967us
I20260709 12:17:53.559677 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:53.560690 21382 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.560758 21598 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.561544 21598 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.564092 21598 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:53.570566 21382 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:53.317049 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 254531us
I20260709 12:17:53.572521 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:53.574157 21560 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:53.341184 21508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.007s
I20260709 12:17:53.415807 21527 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.574914 21655 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:53.575191 21508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.007s	sys 0.011s
W20260709 12:17:53.460657 21377 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.508s	user 0.000s	sys 0.009s
I20260709 12:17:53.575639 18297 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:53.327812 21517 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.836s	user 0.005s	sys 0.005s
I20260709 12:17:53.576399 21517 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:53.388656 21295 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.685s	user 0.003s	sys 0.007s
I20260709 12:17:53.577284 21527 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.577430 21295 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.578611 21517 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:53.329883 21666 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.463s	user 0.003s	sys 0.001s
I20260709 12:17:53.579317 21700 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.579586 21295 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:53.322407 21622 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.849s	user 0.005s	sys 0.005s
W20260709 12:17:53.421854 21640 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.001s
I20260709 12:17:53.579916 21700 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.328275 21654 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:53.387776 21522 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.993s	user 0.012s	sys 0.001s
I20260709 12:17:53.580293 21640 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.581138 21522 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:17:53.460857 21526 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.581974 21700 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.582983 18441 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:53.583760 18442 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:53.586295 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54770us
I20260709 12:17:53.588032 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:53.589910 18370 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:53.590361 21640 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.591015 21577 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:53.591578 21662 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.592937 21577 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.593706 21577 consensus_queue.cc:1243] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:53.598991 18370 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:53.599735 19035 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:17:53.621539 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 79092.8us
I20260709 12:17:53.621527 18370 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:53.623713 18370 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:53.597482 21687 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:53.218366 21398 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:53.644177 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71730.3us
I20260709 12:17:53.631703 18373 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:53.650729 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 91132.1us
I20260709 12:17:53.658526 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:53.659287 19044 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.660846 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:53.660965 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:53.661506 19043 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:53.327572 21702 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:53.588496 21671 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:53.664093 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:53.627861 21628 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.131s	user 0.006s	sys 0.006s
I20260709 12:17:53.576398 21377 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.662564 21522 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.665172 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:53.512928 21631 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:53.667102 19035 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:53.621423 21577 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.672677 19039 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:53.631192 21654 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:53.673408 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:53.654873 21540 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.644522 21687 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:53.674315 18266 leader_election.cc:400] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:53.699159 18266 leader_election.cc:403] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:53.670917 21398 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.666296 21625 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:53.696713 21704 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.704154 21704 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.704483 21704 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.704420 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:53.673810 21671 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.713040 21671 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.654484 21682 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:17:53.715602 21682 consensus_queue.cc:1243] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:53.700178 21540 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.718770 21540 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.672895 21709 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.719609 18443 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:53.720033 18443 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:53.719761 21709 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.720852 18443 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.662984 21542 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:17:53.725325 19044 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:53.676370 21577 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:53.729103 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54940.7us
I20260709 12:17:53.728984 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:53.730659 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:53.689582 21526 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.703271 21522 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.731966 18443 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.732753 19044 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:53.680253 21654 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:53.735685 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 183183us
I20260709 12:17:53.722036 21625 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:53.737197 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:53.672804 21695 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:17:53.736281 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:53.741634 18339 leader_election.cc:336] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.771s (SENT)
I20260709 12:17:53.742990 19042 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:53.679322 21640 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:53.676585 21628 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.751525 18449 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:53.730158 21295 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6.
I20260709 12:17:53.752734 21713 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:53.753304 21713 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:53.753556 21578 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.754448 21628 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:53.754217 21713 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.731426 21577 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:53.749531 21652 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.756s	user 0.010s	sys 0.000s
I20260709 12:17:53.744834 21707 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 12:17:53.741580 21550 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.001s
W20260709 12:17:53.747082 18339 leader_election.cc:336] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.115s (SENT)
I20260709 12:17:53.731736 21517 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.758042 18443 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:53.782235 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2214) took 1044 ms. Trace:
I20260709 12:17:53.782609 18443 rpcz_store.cc:276] 0709 12:17:52.737378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.794367 (+ 56989us) service_pool.cc:224] Handling call
0709 12:17:52.794695 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.719516 (+924821us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54001a3ea0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7e1c0 00007f3c68e7f231 00007f3c68e7bd8f 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c
0709 12:17:53.781981 (+ 62465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.782176 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708941696}
I20260709 12:17:53.673349 21702 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.758105 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:53.745321 21703 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.743424 21522 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.761186 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:53.759330 21295 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:53.697835 21696 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:17:53.761708 21652 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.762423 21709 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:53.763895 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:53.764361 21704 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.766371 21526 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.766095 21625 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.727855 21398 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.744043 21654 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:53.773757 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:53.774191 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:53.783473 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 110151us
I20260709 12:17:53.829051 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:53.812870 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54841.8us
I20260709 12:17:53.823922 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2200) took 1289 ms. Trace:
I20260709 12:17:53.783754 21542 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:53.815044 21699 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:53.835875 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:53.837019 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:53.839885 18448 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:53.834084 18442 rpcz_store.cc:276] 0709 12:17:52.534600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.565879 (+ 31279us) service_pool.cc:224] Handling call
0709 12:17:53.823711 (+1257832us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:53.823851 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338882816}
I20260709 12:17:53.844579 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:53.845652 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:53.846268 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:53.846910 18299 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:53.822068 21295 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:53.777776 21550 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.844441 18366 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:53.810672 21517 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:53.846997 18291 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:53.995630 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:53.849412 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:53.942094 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:53.982942 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:53.883136 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.996462 18372 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:53.972533 18293 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:53.972667 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:53.972096 18292 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:17:53.987521 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.999692 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 252625us
I20260709 12:17:53.848840 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.000726 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:53.996706 18295 raft_consensus.cc:2368] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7.
I20260709 12:17:53.847370 21709 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:53.860935 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61571.8us
I20260709 12:17:54.001435 18300 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:17:54.027549 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 193742us
I20260709 12:17:54.002059 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 631) took 1330 ms. Trace:
W20260709 12:17:54.002561 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 170171us
I20260709 12:17:54.028680 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.003897 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1796) took 1052 ms. Trace:
W20260709 12:17:54.005131 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.992756 21704 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.028221 18371 rpcz_store.cc:276] 0709 12:17:52.671493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.671721 (+   228us) service_pool.cc:224] Handling call
0709 12:17:52.672125 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.782792 (+110667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.782824 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:53.661759 (+878935us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b2c0005b678. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:53.662155 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 12:17:53.733361 (+ 71206us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:53.733415 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.733505 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.981125 (+247620us) raft_consensus.cc:1685] finished
0709 12:17:53.981153 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:54.001998 (+ 20845us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34287,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"spinlock_wait_cycles":6635392,"thread_start_us":167510,"threads_started":2,"wal-append.queue_time_us":164449}
I20260709 12:17:54.041987 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.043622 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.005947 21550 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.008421 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:54.009001 21712 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.337s	user 0.001s	sys 0.009s
I20260709 12:17:54.009104 21702 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:54.009622 21707 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:54.011520 21668 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:54.016224 21527 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.062513 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:54.063184 19044 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.016100 21577 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.008488 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:54.000908 21652 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:53.974347 21522 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.020092 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2217) took 1281 ms. Trace:
I20260709 12:17:54.021559 21654 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:53.940761 21563 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.972921 21377 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:53.941337 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.077531 18370 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:54.066519 21527 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.078716 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:54.079474 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 50865.7us
I20260709 12:17:54.079947 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.024181 18296 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:54.029500 18445 rpcz_store.cc:276] 0709 12:17:52.967752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.967877 (+   125us) service_pool.cc:224] Handling call
0709 12:17:52.968204 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.968539 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.968565 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:53.762978 (+794413us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:17:53.763379 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 12:17:53.845003 (+ 81624us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:53.845048 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.845110 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:54.003612 (+158502us) raft_consensus.cc:1685] finished
0709 12:17:54.003632 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:54.003842 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":441,"raft.queue_time_us":90431,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":3549,"spinlock_wait_cycles":5025024,"thread_start_us":81282,"threads_started":1,"wal-append.queue_time_us":238548}
I20260709 12:17:54.031242 21709 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:54.011538 21710 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:17:54.084061 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:54.048822 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.840448 21396 raft_consensus.cc:2764] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:54.052623 18375 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.056242 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.056957 21642 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:54.057951 21565 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:54.052026 21497 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:17:54.059846 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:54.094712 19040 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.840453 21625 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.829661 21526 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.066817 18444 rpcz_store.cc:276] 0709 12:17:52.738306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.810440 (+ 72134us) service_pool.cc:224] Handling call
0709 12:17:52.911172 (+100732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.916844 (+  5672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.019858 (+1103014us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b5400424350. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:54.020031 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179323776}
I20260709 12:17:53.847299 21542 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.072094 19044 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:54.099864 18371 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:54.081881 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.047689 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.062459 21668 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:54.087215 18292 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:54.100983 19041 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:54.089967 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:54.102377 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.103623 18296 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:54.096032 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:54.059470 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.104465 18370 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.100862 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.113231 18298 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:54.083907 21622 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:17:54.114434 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 630) took 1444 ms. Trace:
I20260709 12:17:54.114353 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.114879 18369 rpcz_store.cc:276] 0709 12:17:52.670366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.670502 (+   136us) service_pool.cc:224] Handling call
0709 12:17:52.679448 (+  8946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.936674 (+257226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.936719 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:52.936773 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:52.936823 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.114183 (+1177360us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54000f9954. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:54.114245 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:54.114378 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591047936}
I20260709 12:17:54.116938 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.111567 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.134234 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.120895 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:54.137169 21663 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:17:54.137282 18292 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:54.139374 18293 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:54.121129 21377 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:54.141608 21377 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:54.142820 21377 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:54.143882 18449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2192) took 1612 ms (client timeout 1597 ms). Trace:
W20260709 12:17:54.144308 18449 rpcz_store.cc:269] 0709 12:17:52.531819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.532873 (+  1054us) service_pool.cc:224] Handling call
0709 12:17:54.143696 (+1610823us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003b24d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:54.143817 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750995328}
I20260709 12:17:54.145380 21680 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:54.145826 21680 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:54.146741 21680 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.149216 21707 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:54.149736 21555 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.151608 21635 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:54.152160 21635 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:54.152456 21654 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.153015 21635 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:54.153736 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:54.154007 21635 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.097748 21565 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.159966 21565 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:54.167906 19044 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:17:54.168864 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 140740us
I20260709 12:17:54.169903 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:54.175446 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54677us
I20260709 12:17:54.176429 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:54.179958 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.115825 21642 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.149739 21707 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.198563 21537 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:54.209026 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.175310 21527 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:54.181159 21522 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:54.198799 21622 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.214649 18370 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
W20260709 12:17:54.202666 21723 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:17:54.218189 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.123011 21295 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.213928 19040 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:54.211438 21707 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:54.198977 21565 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:17:54.233304 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:54.234072 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.229739 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.220046 21678 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.223027 21622 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.229298 21522 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:54.248389 21654 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:54.253530 18370 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:54.221000 21708 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.231302 21663 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:54.284313 21719 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:17:54.284466 21522 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:54.291638 21726 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:17:54.292641 21542 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:54.292831 21577 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:54.251322 21678 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.254200 21667 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:17:54.298578 21709 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:54.295097 21678 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:54.303978 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 701) took 1816 ms. Trace:
I20260709 12:17:54.304359 18441 rpcz_store.cc:276] 0709 12:17:52.487904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.489294 (+  1390us) service_pool.cc:224] Handling call
0709 12:17:52.489568 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.583485 (+1093917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.303733 (+720248us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000d9654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:54.303906 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514069760}
I20260709 12:17:54.305728 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.306442 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46944 (request call id 21) took 1404 ms. Trace:
I20260709 12:17:54.256814 21531 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:17:54.251013 21563 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:54.307605 21563 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.307998 21563 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:54.256795 21578 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.308485 21578 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.310626 21578 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.206202 21668 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.311870 21668 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.313887 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.314038 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.314497 18442 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
W20260709 12:17:54.315379 21573 consensus_queue.cc:1175] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "177254018ba7448aa2b9110c9c9be9e3" responder_term: 6 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:54.316146 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2144) took 1145 ms. Trace:
I20260709 12:17:54.316542 18297 rpcz_store.cc:276] 0709 12:17:53.170610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.170717 (+   107us) service_pool.cc:224] Handling call
0709 12:17:54.315933 (+1145216us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400090454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:54.316076 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923876608}
I20260709 12:17:52.720332 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:54.306784 18440 rpcz_store.cc:276] 0709 12:17:52.902387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.903301 (+   914us) service_pool.cc:224] Handling call
0709 12:17:54.306217 (+1402916us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e82735 00007f3c6de42d29 00007f3c673c11d4 00007f3c673c1162 00007f3c673c1091 00007f3c673c100c 00007f3c673c02c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:54.306376 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565308800}
I20260709 12:17:54.318921 21573 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.319527 21573 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.324656 21709 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.325320 21602 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.327497 18291 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:54.329216 18291 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:17:54.330870 18291 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:54.331027 21398 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:54.332317 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.334255 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.282051 18370 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:54.325165 21709 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.347270 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.253052 19040 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:54.337136 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.251137 21622 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.306967 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.346668 18297 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:54.358495 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.347352 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.364130 18297 pending_rounds.cc:85] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 1
I20260709 12:17:54.364666 18297 pending_rounds.cc:107] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:54.364996 18297 raft_consensus.cc:2889] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.365063 18375 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:54.366752 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.369093 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.370024 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.371675 18293 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:54.373174 18293 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:54.363968 18370 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.375176 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.375855 18441 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.376127 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.376803 18293 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.378734 18447 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:17:54.380511 18197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 293) took 16376 ms (client timeout 15000 ms). Trace:
I20260709 12:17:54.380591 18370 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.369798 18440 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:54.385504 18440 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.386898 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.387897 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.388433 18292 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:54.388374 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:54.391156 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 155073us
I20260709 12:17:54.392395 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.343391 21654 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:54.399587 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.331389 21707 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.416433 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.417855 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.418967 18447 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:17:54.380993 18197 rpcz_store.cc:269] 0709 12:17:38.004356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.004622 (+   266us) service_pool.cc:224] Handling call
0709 12:17:54.380448 (+16375826us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:48.571637 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:48.572260 (+   623us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:48.572310 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:17:48.572326 (+    16us) tablet.cc:662] Decoding operations
0709 12:17:48.573694 (+  1368us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:48.573773 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:48.573801 (+    28us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:17:48.575050 (+  1249us) tablet.cc:701] Row locks acquired
0709 12:17:48.575098 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:17:48.611725 (+ 36627us) write_op.cc:270] Start()
0709 12:17:48.630099 (+ 18374us) write_op.cc:276] Timestamp: P: 1783599468575192 usec, L: 0
0709 12:17:48.630129 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:17:48.686814 (+ 56685us) log.cc:844] Serialized 6313 byte log entry
0709 12:17:49.328370 (+641556us) op_driver.cc:464] REPLICATION: finished
0709 12:17:49.672951 (+344581us) write_op.cc:301] APPLY: starting
0709 12:17:49.686869 (+ 13918us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:49.798031 (+111162us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:49.822178 (+ 24147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:49.973235 (+151057us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:49.973719 (+   484us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:17:49.973791 (+    72us) write_op.cc:312] APPLY: finished
0709 12:17:49.974373 (+   582us) log.cc:844] Serialized 113 byte log entry
0709 12:17:49.974603 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:49.974810 (+   207us) write_op.cc:454] Released schema lock
0709 12:17:49.974899 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:50.000840 (+ 25941us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:50.007479 (+  6639us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":671446,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":302840,"apply.run_cpu_time_us":5880,"apply.run_wall_time_us":329149,"num_ops":16,"prepare.queue_time_us":171763,"prepare.run_cpu_time_us":24142,"prepare.run_wall_time_us":291663,"raft.queue_time_us":331584,"raft.run_cpu_time_us":48874,"raft.run_wall_time_us":289397,"replication_time_us":631208,"thread_start_us":644048,"threads_started":4,"wal-append.queue_time_us":210957,"wal-append.run_cpu_time_us":63090,"wal-append.run_wall_time_us":2077411}]]}
I20260709 12:17:54.422835 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:54.424532 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.426411 18370 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:54.429337 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:54.430194 18370 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:54.429337 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.431149 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.431815 18445 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:54.434505 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.435576 18296 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:54.390880 21555 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.436921 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.437865 18449 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:54.402405 21241 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.405501 21708 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.443763 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:54.444568 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet b81b910b60b0422f9db4c0164608fcbb: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:17:54.447441 18442 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:54.409827 21663 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.456723 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.415292 21565 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.468020 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.405670 21542 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:17:54.468937 19040 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:54.481750 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.486274 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.462215 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.427918 21573 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.426724 21499 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:17:54.460415 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.449988 21628 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.498409 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.503876 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.506310 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.498668 21577 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:54.512666 21709 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.480530 21565 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.516387 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.517124 21565 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.453804 21555 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.518623 18445 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 5.
I20260709 12:17:54.519222 21555 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.457049 21241 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.521528 21241 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:54.521421 21628 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.522127 21652 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.486868 21654 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.520956 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.524081 18445 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.524765 21707 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.525976 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:54.526731 19040 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:17:54.527143 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.523622 21628 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.507350 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.545120 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.486691 21542 consensus_queue.cc:1243] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:54.506863 18442 raft_consensus.cc:2376] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8. Re-sending same reply.
I20260709 12:17:54.505641 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:54.554934 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.558257 18440 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:54.566727 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:54.568012 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 104274us
I20260709 12:17:54.568161 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.570559 19044 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:54.574700 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:54.593523 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.548167 21295 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.628074 19035 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:54.576867 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.478564 21663 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:54.634855 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.635398 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.600643 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.637185 18374 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:54.522617 21652 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.556139 21709 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.559756 21542 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:54.631054 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:17:54.595386 18369 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:54.610307 21625 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.000s
I20260709 12:17:54.576362 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.650584 18442 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:54.640657 18293 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:17:54.608474 21731 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:17:54.548197 21398 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:54.655702 18447 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:54.568439 21377 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.562331 21707 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:17:54.656333 21377 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.656844 21377 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.657358 21707 consensus_queue.cc:1243] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:54.649055 18445 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:54.649216 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.660358 21654 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:54.663002 18441 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:54.567656 18440 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:17:54.611392 21577 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.630697 21295 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.660980 21654 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:54.664729 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.668730 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:54.551892 21565 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:17:54.670480 18441 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:17:54.606663 21748 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:17:54.672966 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.673628 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.673946 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.670583 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.675275 18442 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:54.593758 21555 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:17:54.703033 18440 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:54.706037 18440 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.704192 18441 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:54.701254 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:54.708637 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.709614 21677 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:54.711653 21654 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.668610 21577 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:54.716219 21577 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.715495 21687 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.717103 21687 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.654176 21542 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.721597 21542 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:54.722468 18373 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.724725 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:54.725476 18442 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:54.659723 21522 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:17:54.716246 21642 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:54.728844 21642 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:54.729993 21642 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.731297 21642 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.731668 21555 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.732318 21555 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:54.733433 21555 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.734190 21622 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.665573 21573 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.735500 18443 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:17:54.708750 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.709040 18445 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:54.669651 21625 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.741148 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.673007 21295 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.735658 21550 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:54.739343 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 70696.8us
I20260709 12:17:54.747498 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.707736 18440 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:54.753511 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.753624 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.754159 18292 raft_consensus.cc:2376] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
I20260709 12:17:54.757619 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.764441 18445 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:17:54.775456 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.804658 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.806644 18292 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:54.746217 21677 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:54.805580 19040 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:17:54.808260 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.748215 21722 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:54.811430 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.815011 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.739928 21598 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:17:54.819995 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.759905 21727 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:54.821995 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.822577 18298 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 5: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:54.822566 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.765944 21735 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:54.748817 21657 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:17:54.823559 18292 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:17:54.746718 21573 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.818233 21677 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.832051 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.753211 21537 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:17:54.809919 21528 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.815567 21748 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:54.749884 21625 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.804196 21563 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.755352 21550 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.858127 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.833012 21754 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:54.863082 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:54.852452 21677 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.879302 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.883634 18375 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:54.880268 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:54.885229 21757 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:54.891192 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.886754 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.859071 21760 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.852871 21748 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:54.901095 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.896801 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.897476 18292 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:54.902060 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:54.854498 21528 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.923136 18373 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:54.923681 21772 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:54.924016 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.924837 21772 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:54.925376 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.920574 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:54.926726 21772 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:54.926741 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.930590 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.846340 21735 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.892136 21598 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.948568 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.931496 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:17:54.917856 21668 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.000s
I20260709 12:17:54.875756 21754 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.949671 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:54.951031 18442 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:17:54.898584 21652 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.951774 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:54.953763 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57061.6us
I20260709 12:17:54.899935 21660 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:54.929044 21622 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:54.928884 21537 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.960917 19035 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.935372 21729 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.958595 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:54.959463 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:54.965900 18373 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:17:54.966594 21537 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:54.957370 21598 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:54.985255 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.988582 21776 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.989104 21776 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.989468 21776 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.989204 21598 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:54.991442 18373 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.992939 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:54.993649 18298 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:54.962679 21757 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:54.997462 18366 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:54.994377 19035 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:54.994493 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:55.000000 18298 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:55.002120 21537 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:55.004612 21754 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:55.007472 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:55.009274 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:55.009356 21377 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.009502 21776 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.009603 21528 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.010365 18298 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
W20260709 12:17:55.011101 21753 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:17:55.000265 21573 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.011224 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:55.016393 21625 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:55.016664 18292 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:55.017915 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:55.020648 21377 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.022472 21577 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.023490 18291 raft_consensus.cc:2368] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:55.023861 18369 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:55.024509 21563 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:55.026840 21708 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.559s	user 0.005s	sys 0.005s
I20260709 12:17:55.013290 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:54.901013 21760 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.033918 18294 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:54.999068 21729 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:17:55.045763 18298 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:55.029947 21708 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:55.050868 21767 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:17:55.051432 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.052073 21680 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:55.056145 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 132227us
I20260709 12:17:55.056360 18366 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:55.044397 21577 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:55.047492 21729 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.058625 18298 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:55.059155 21563 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:55.059394 21573 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:55.060431 21760 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.060606 21748 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:55.064466 21730 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.072099 21730 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.073153 21526 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.074617 21730 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.075726 21295 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.077646 21295 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.078644 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:55.079890 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:17:55.081163 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.082206 21623 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.086126 21709 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.087476 18296 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 12:17:55.090471 18296 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 7. Ops: [5.3-5.3]
I20260709 12:17:55.091773 18293 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:55.092360 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:55.093360 18293 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:55.093513 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:55.094652 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:55.096910 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.097225 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:55.097820 18446 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:55.097908 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.098325 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:55.099517 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:55.097949 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.086971 21709 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.100037 21664 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.105450 21709 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.107667 21709 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.062300 21729 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:17:55.111799 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:55.113183 21625 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:55.114239 21625 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:55.111387 21748 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:55.116449 21748 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:55.117324 21748 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.111246 21642 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:55.119727 21642 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.120066 21642 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:55.113606 18373 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:55.063537 21550 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:55.133488 21730 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:55.133890 21730 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.134050 21517 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.134238 21730 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.134788 21757 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:55.137738 21241 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:55.138998 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 862) took 1918 ms. Trace:
I20260709 12:17:55.139389 19042 rpcz_store.cc:276] 0709 12:17:53.220400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.220583 (+   183us) service_pool.cc:224] Handling call
0709 12:17:53.236312 (+ 15729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.746630 (+510318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.138779 (+1392149us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54008205d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:55.138939 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164740992}
W20260709 12:17:55.143926 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.146639 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:55.148949 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:17:55.493204 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.524261 18373 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.210253 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.111584 21708 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:17:55.515801 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.269801 18446 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:55.301558 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 142399us
I20260709 12:17:55.535468 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:55.514230 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:55.526481 18445 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
W20260709 12:17:55.542197 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.206845 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:55.573203 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:55.594326 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.563s (SENT)
I20260709 12:17:55.594992 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:17:55.594919 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 80772us
I20260709 12:17:55.606063 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:55.608666 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:55.610955 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:55.530735 21754 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.616554 18292 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:55.445281 21757 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.614605 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:55.615595 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.624396 18445 raft_consensus.cc:2410] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:55.616670 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:55.633693 18373 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:55.497138 21760 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 12:17:55.613122 21687 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.637663 21774 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:55.628577 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:55.648875 21766 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.633s	user 0.005s	sys 0.007s
I20260709 12:17:55.649502 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:55.650712 21577 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:55.651144 21577 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.652545 21577 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:55.656234 21760 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:55.666819 21680 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.000s
I20260709 12:17:55.298830 21578 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.321s
I20260709 12:17:55.613435 21668 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.607832 21295 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:55.672760 21623 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.003s
I20260709 12:17:55.617636 21655 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:55.620071 21690 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.204815 21677 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.625296 21754 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.637408 21526 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:17:55.675892 21777 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:17:55.637248 21573 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:17:55.677672 21531 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:55.444644 21667 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.312s
W20260709 12:17:55.679849 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65324.6us
I20260709 12:17:55.707144 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2223) took 1282 ms. Trace:
I20260709 12:17:55.612394 21708 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.748385 18444 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:17:55.796025 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:55.693370 21709 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:55.830353 18443 rpcz_store.cc:276] 0709 12:17:54.424858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.426369 (+  1511us) service_pool.cc:224] Handling call
0709 12:17:54.427269 (+   900us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:54.736039 (+308770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.706776 (+970737us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540035fa54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:55.707061 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":707929472}
I20260709 12:17:55.834457 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:55.641348 21757 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:55.827423 21730 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 12:17:55.845666 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.857198 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:55.854702 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.867220 18445 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:55.846488 18443 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:55.827714 21531 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.839205 21709 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.839632 21682 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.870545 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:55.875605 18443 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.860731 21696 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:55.875844 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.843716 21577 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.844077 21668 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:17:55.841087 21708 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.857040 21730 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.876976 18445 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:17:55.869174 21680 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:55.888919 18442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.013s	sys 0.004s
I20260709 12:17:55.888218 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2155) took 1525 ms. Trace:
W20260709 12:17:55.889428 18442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.013s	sys 0.004s
I20260709 12:17:55.888325 18298 raft_consensus.cc:1217] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-5.3]   Dedup: 5.3->[]
I20260709 12:17:55.891129 18298 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:55.885710 21616 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:55.894850 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1736) took 1083 ms. Trace:
I20260709 12:17:55.895210 18291 rpcz_store.cc:276] 0709 12:17:54.811204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.814720 (+  3516us) service_pool.cc:224] Handling call
0709 12:17:55.890989 (+1076269us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540040e054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:55.894791 (+  3802us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477658368}
I20260709 12:17:55.872609 21623 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.874914 21709 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.874061 21531 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:17:55.872112 21642 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.752s	user 0.008s	sys 0.002s
I20260709 12:17:55.893338 21616 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:55.883090 21696 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.912986 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:55.918499 18445 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.911815 21623 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:55.915041 21782 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:55.914907 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:55.922988 18291 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:55.917371 18443 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:17:55.920046 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.919551 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:55.893573 21680 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.925688 21680 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.926023 18372 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:55.931754 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:55.932865 18373 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:55.892509 21687 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.885313 21682 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:55.937608 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.941190 18443 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:55.902285 21628 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.915975 21781 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:17:55.916091 21757 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:55.916900 21669 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:55.954500 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:55.904387 21748 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:55.938292 21295 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:55.957603 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:55.935318 21787 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:55.923070 21696 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:55.961090 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:55.961895 18295 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.923626 21777 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.966683 18367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.032s	sys 0.048s
I20260709 12:17:55.967018 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:55.967881 18367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.032s	sys 0.048s
I20260709 12:17:55.969141 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 719) took 1438 ms. Trace:
I20260709 12:17:55.969571 18443 rpcz_store.cc:276] 0709 12:17:54.530515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.959971 (+1429456us) service_pool.cc:224] Handling call
0709 12:17:55.968492 (+  8521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.968723 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:55.968767 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:55.968822 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:55.968867 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.968938 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.969074 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.944368 21687 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.973085 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:55.973714 18443 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:55.974560 21504 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:55.974920 21504 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:55.975781 21504 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.976473 21623 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:55.976809 21504 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.977875 18295 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:55.978318 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 648) took 1579 ms. Trace:
I20260709 12:17:55.978713 18375 rpcz_store.cc:276] 0709 12:17:54.398536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.401168 (+  2632us) service_pool.cc:224] Handling call
0709 12:17:54.401859 (+   691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:54.884005 (+482146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.978098 (+1094093us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400114c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:55.978258 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105142016}
I20260709 12:17:55.979965 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2221) took 1580 ms. Trace:
I20260709 12:17:55.980304 18444 rpcz_store.cc:276] 0709 12:17:54.399454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.399563 (+   109us) service_pool.cc:224] Handling call
0709 12:17:55.979813 (+1580250us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001a5c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:55.979909 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3061053696}
I20260709 12:17:55.981642 18444 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:55.982162 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 720) took 1451 ms. Trace:
I20260709 12:17:55.982435 18444 rpcz_store.cc:276] 0709 12:17:54.530875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.981179 (+1450304us) service_pool.cc:224] Handling call
0709 12:17:55.981483 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.981966 (+   483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.982120 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.955243 21677 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:55.947379 21616 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:55.984580 21616 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:55.985592 21729 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.877s	user 0.009s	sys 0.001s
I20260709 12:17:55.985987 21729 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:55.987593 18294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1726) took 2142 ms (client timeout 1779 ms). Trace:
I20260709 12:17:55.987993 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:55.988150 18294 rpcz_store.cc:269] 0709 12:17:53.845196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.845363 (+   167us) service_pool.cc:224] Handling call
0709 12:17:55.987390 (+2142027us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000a9bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:55.987508 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709795072}
I20260709 12:17:55.989238 18296 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:17:55.989814 21535 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:55.990252 21535 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:55.992874 21644 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:17:55.994536 21644 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:17:55.994836 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:55.996140 18375 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:17:55.997304 18449 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:55.998585 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:55.999068 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:56.000622 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1849) took 1102 ms. Trace:
I20260709 12:17:56.003712 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1735) took 1192 ms. Trace:
I20260709 12:17:56.021158 18298 rpcz_store.cc:276] 0709 12:17:54.810903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.062918 (+252015us) service_pool.cc:224] Handling call
0709 12:17:55.888185 (+825267us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:55.891565 (+  3380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.003459 (+111894us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400275bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:56.003632 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868489088}
I20260709 12:17:55.889552 18297 rpcz_store.cc:276] 0709 12:17:54.362662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.363426 (+   764us) service_pool.cc:224] Handling call
0709 12:17:54.363852 (+   426us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:54.370568 (+  6716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.370604 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:54.371046 (+   442us) log.cc:844] Serialized 78 byte log entry
0709 12:17:54.499593 (+128547us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:54.757761 (+258168us) log.cc:844] Serialized 14 byte log entry
0709 12:17:54.765817 (+  8056us) log.cc:844] Serialized 14 byte log entry
0709 12:17:55.876595 (+1110778us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b1800436c60. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c68a32d70 00007f3c68a26293 00007f3c68a1bcea 00007f3c68a1c23a 00007f3c68e86a2a 00007f3c68e960bf 00007f3c68e96038
0709 12:17:55.884765 (+  8170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.884839 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:55.887983 (+  3144us) raft_consensus.cc:1685] finished
0709 12:17:55.888002 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.888160 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1081335,"prepare.queue_time_us":71750,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":1130,"raft.queue_time_us":1835,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":54375,"spinlock_wait_cycles":8311552,"thread_start_us":300066,"threads_started":2,"wal-append.queue_time_us":182609}
I20260709 12:17:55.987502 21655 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.936041 21690 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:17:56.009074 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:55.922853 21642 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.011075 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1741) took 1165 ms. Trace:
I20260709 12:17:56.033376 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:56.016145 18445 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:55.973131 21754 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.020679 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 722) took 1471 ms. Trace:
I20260709 12:17:55.961901 21787 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:55.967710 21295 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:55.994628 21680 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.015862 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:56.034225 18297 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:55.950039 21398 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:17:56.032335 18296 rpcz_store.cc:276] 0709 12:17:54.826041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.091115 (+265074us) service_pool.cc:224] Handling call
0709 12:17:55.988939 (+897824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.989585 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.991095 (+  1510us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54000a9bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:55.991208 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036329856}
I20260709 12:17:56.036449 18449 rpcz_store.cc:276] 0709 12:17:54.549804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.996849 (+1447045us) service_pool.cc:224] Handling call
0709 12:17:55.997137 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.020459 (+ 23322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.020629 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.946224 21628 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.021035 18366 rpcz_store.cc:276] 0709 12:17:54.897946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.998235 (+100289us) service_pool.cc:224] Handling call
0709 12:17:54.998467 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.056750 (+ 58283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.000321 (+943571us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b54002df0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:56.000546 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128968448}
W20260709 12:17:56.024437 21781 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:17:56.049055 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:56.104712 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.108783 18443 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:55.961539 21748 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:56.139714 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90745.7us
I20260709 12:17:56.141074 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:56.142416 18445 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.143242 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 723) took 1588 ms. Trace:
I20260709 12:17:56.143651 18445 rpcz_store.cc:276] 0709 12:17:54.555099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.034575 (+1479476us) service_pool.cc:224] Handling call
0709 12:17:56.142194 (+107619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.142941 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.143160 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.146485 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 574) took 1011 ms. Trace:
I20260709 12:17:56.146974 18296 rpcz_store.cc:276] 0709 12:17:55.135349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.048876 (+913527us) service_pool.cc:224] Handling call
0709 12:17:56.145874 (+ 96998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.146080 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.146119 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.146171 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:56.146223 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.146290 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.146412 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.960148 21669 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.152807 18297 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:56.150329 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 571) took 1269 ms. Trace:
I20260709 12:17:56.151839 18449 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:56.156899 18296 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 12:17:56.157299 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 724) took 1597 ms. Trace:
I20260709 12:17:56.157796 18449 rpcz_store.cc:276] 0709 12:17:54.560214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.103541 (+1543327us) service_pool.cc:224] Handling call
0709 12:17:56.151608 (+ 48067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.153946 (+  2338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.157205 (+  3259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.153795 18293 rpcz_store.cc:276] 0709 12:17:54.881437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.094358 (+212921us) service_pool.cc:224] Handling call
0709 12:17:55.102065 (+  7707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:55.102475 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:55.102521 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:55.109362 (+  6841us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.218822 (+109460us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:55.219197 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 12:17:55.297858 (+ 78661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.297935 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.150062 (+852127us) raft_consensus.cc:1685] finished
0709 12:17:56.150088 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.150261 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21492,"prepare.queue_time_us":77232,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":78843,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":6267,"thread_start_us":109134,"threads_started":1,"wal-append.queue_time_us":720012}
I20260709 12:17:55.966799 21777 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.160199 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 727) took 1564 ms. Trace:
I20260709 12:17:56.103628 21677 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.160821 18443 rpcz_store.cc:276] 0709 12:17:54.595818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.116703 (+1520885us) service_pool.cc:224] Handling call
0709 12:17:56.159162 (+ 42459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.159400 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.159548 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.159609 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:56.159664 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.159736 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.160133 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.160382 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 577) took 1024 ms. Trace:
I20260709 12:17:56.103988 21708 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.164945 18296 rpcz_store.cc:276] 0709 12:17:55.136041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.151545 (+1015504us) service_pool.cc:224] Handling call
0709 12:17:56.151851 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.160134 (+  8283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.160326 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.036418 21655 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.152352 21722 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:55.991218 21535 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.230069 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 728) took 1605 ms. Trace:
I20260709 12:17:56.230630 18445 rpcz_store.cc:276] 0709 12:17:54.624974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.164814 (+1539840us) service_pool.cc:224] Handling call
0709 12:17:56.229414 (+ 64600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.229649 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.229686 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.229744 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:56.229790 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.229860 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.229999 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.156770 21667 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:56.219491 21730 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.181031 21729 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.233815 21677 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.314980 21708 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:56.325187 21777 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:56.326064 21786 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:17:56.330189 21754 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:56.335201 21727 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:56.318168 21535 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:56.336931 21655 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.337087 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 573) took 1202 ms. Trace:
I20260709 12:17:56.228809 21577 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.340883 18449 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 12:17:56.358963 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.426936 18449 pending_rounds.cc:85] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:17:56.383575 21687 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.343096 21677 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.397017 21748 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:17:56.398394 21789 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:17:56.400964 21628 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.402506 21669 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.404256 21680 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.311338 21667 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.299348 21676 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.415939 18264 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.010s	sys 0.016s
W20260709 12:17:56.461973 18264 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.010s	sys 0.016s
W20260709 12:17:56.462383 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 945713us
I20260709 12:17:56.424969 18297 rpcz_store.cc:276] 0709 12:17:55.135084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.047767 (+912683us) service_pool.cc:224] Handling call
0709 12:17:56.048174 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.153331 (+105157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.336720 (+183389us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54001cd9d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:56.336999 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239983872}
W20260709 12:17:56.469676 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.470120 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.427274 18449 pending_rounds.cc:107] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:17:56.471645 18449 raft_consensus.cc:2889] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:56.472563 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1819) took 1818 ms. Trace:
I20260709 12:17:56.472929 18449 rpcz_store.cc:276] 0709 12:17:54.653963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.340258 (+1686295us) service_pool.cc:224] Handling call
0709 12:17:56.340573 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.472242 (+131669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.472495 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.452261 21677 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:56.437278 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.036s	sys 0.042s
I20260709 12:17:56.417948 21708 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.420387 21727 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:56.425365 21642 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.453862 21577 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:56.322137 21295 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.467002 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:56.476617 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.036s	sys 0.042s
W20260709 12:17:56.418169 21563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.007s	sys 0.016s
I20260709 12:17:56.482121 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:56.425136 21754 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.498351 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:56.483996 21598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.018s	sys 0.017s
I20260709 12:17:56.500550 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:56.478894 21680 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:56.502790 18412 leader_election.cc:400] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
I20260709 12:17:56.503144 18412 leader_election.cc:403] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:56.503783 21642 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7.
W20260709 12:17:56.484867 21563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.007s	sys 0.016s
W20260709 12:17:56.508782 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.008s	sys 0.029s
W20260709 12:17:56.510939 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.008s	sys 0.030s
W20260709 12:17:56.512451 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 519557us
W20260709 12:17:56.513753 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.041s	sys 0.047s
W20260709 12:17:56.527627 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.042s	sys 0.048s
W20260709 12:17:56.528075 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 876104us
I20260709 12:17:56.475317 21676 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.522523 21667 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.528898 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:56.521929 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.500489 21598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.018s	sys 0.017s
I20260709 12:17:56.531039 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 716) took 2001 ms. Trace:
I20260709 12:17:56.536121 18442 rpcz_store.cc:276] 0709 12:17:54.529540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.961763 (+432223us) service_pool.cc:224] Handling call
0709 12:17:54.962618 (+   855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.963046 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.963087 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.963702 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.889788 (+926086us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:55.889831 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.889899 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.530792 (+640893us) raft_consensus.cc:1685] finished
0709 12:17:56.530820 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.530970 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4553,"thread_start_us":859872,"threads_started":1,"wal-append.queue_time_us":1151760}
I20260709 12:17:56.538131 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:56.539197 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:56.484699 21577 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:56.530438 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 374320us
W20260709 12:17:56.558955 18299 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.015s
I20260709 12:17:56.559967 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:17:56.560381 18299 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.008s	sys 0.015s
I20260709 12:17:56.531482 21417 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.530537 21687 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:56.509399 21616 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:17:56.528824 21563 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:56.526443 21680 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:56.570465 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.571300 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:56.571348 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:56.568813 21417 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:56.566437 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.000s	sys 0.040s
I20260709 12:17:56.544183 21669 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:17:56.597865 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.000s	sys 0.040s
I20260709 12:17:56.559012 21526 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:56.597642 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:56.561321 21667 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:56.544385 21777 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:56.614956 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.591792 21776 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.558526 21790 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:56.601835 21417 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:56.639653 21760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.004s	sys 0.038s
I20260709 12:17:56.601642 21667 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.558321 21504 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:17:56.543839 21727 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.558652 21347 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:17:56.628472 21791 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:56.597327 21563 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:56.633060 18265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.021s	sys 0.024s
W20260709 12:17:56.636118 18291 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.018s	sys 0.024s
I20260709 12:17:56.683871 18442 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:17:56.684201 18291 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.018s	sys 0.024s
I20260709 12:17:56.684516 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1821) took 2017 ms. Trace:
W20260709 12:17:56.684779 18265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.021s	sys 0.024s
I20260709 12:17:56.684863 18442 rpcz_store.cc:276] 0709 12:17:54.667336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.662848 (+1995512us) service_pool.cc:224] Handling call
0709 12:17:56.680625 (+ 17777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.684289 (+  3664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.684467 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:56.685499 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.678602 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 180323us
I20260709 12:17:56.650815 21555 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.687146 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:56.668393 21760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.004s	sys 0.038s
I20260709 12:17:56.672576 21667 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:56.674564 21776 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:56.680255 21708 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.629791 21526 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.648641 21777 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.651435 21669 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.683780 21791 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:17:56.713798 21760 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:56.669057 21790 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.720007 21504 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:56.746362 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:56.729686 21802 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:56.743029 21687 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.731025 21673 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:17:56.734758 21800 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:17:56.750643 21760 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.765750 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 721) took 2234 ms. Trace:
I20260709 12:17:56.771095 21676 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.772104 21676 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.772864 21676 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:56.773685 21676 consensus_queue.cc:1243] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:56.757855 21754 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.777722 21790 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.778524 21790 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:56.779177 21790 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.751948 21573 raft_consensus.cc:2764] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:56.724565 21798 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:56.713774 21797 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:17:56.757176 21799 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.762632 21696 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.000s
I20260709 12:17:56.762650 21687 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:56.763346 18447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2230) took 2324 ms (client timeout 1563 ms). Trace:
W20260709 12:17:56.763615 21795 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:17:56.826740 21709 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.909s	user 0.005s	sys 0.007s
I20260709 12:17:56.782279 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1731) took 1975 ms. Trace:
I20260709 12:17:56.823843 21708 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:56.785553 21531 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.000s
I20260709 12:17:56.744418 21805 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:56.755287 21802 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:56.792778 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1823) took 2045 ms. Trace:
I20260709 12:17:56.835502 21577 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.780664 21677 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.828424 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:17:56.837675 18447 rpcz_store.cc:269] 0709 12:17:54.438762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.439586 (+   824us) service_pool.cc:224] Handling call
0709 12:17:56.763047 (+2323461us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54002ab054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:56.763212 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436523904}
I20260709 12:17:56.766161 18444 rpcz_store.cc:276] 0709 12:17:54.531178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.983100 (+1451922us) service_pool.cc:224] Handling call
0709 12:17:56.763699 (+780599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.763882 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.763909 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.763949 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:56.763985 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.764034 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.765569 (+  1535us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54002ab050. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:56.765689 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729048960}
W20260709 12:17:56.785163 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.872023 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.874819 19038 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:56.875032 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:56.853664 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 282379us
I20260709 12:17:56.853266 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1824) took 2087 ms. Trace:
I20260709 12:17:56.876609 18442 rpcz_store.cc:276] 0709 12:17:54.766399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.827540 (+2061141us) service_pool.cc:224] Handling call
0709 12:17:56.852605 (+ 25065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.852827 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.852863 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.852918 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:56.852967 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.853036 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.853197 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.851141 18292 rpcz_store.cc:276] 0709 12:17:54.807164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.017346 (+210182us) service_pool.cc:224] Handling call
0709 12:17:55.017591 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.617131 (+599540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.782064 (+1164933us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54002814d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:56.782222 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376903552}
I20260709 12:17:56.856290 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:56.856442 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 288415us
I20260709 12:17:56.857744 18373 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
W20260709 12:17:56.882179 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53869.6us
I20260709 12:17:56.756445 21776 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:56.827803 21678 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.889019 21803 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:17:56.889274 21678 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.890223 21648 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:56.859556 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:56.860822 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:56.861569 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 301662us
I20260709 12:17:56.862164 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:56.863086 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 729) took 2230 ms. Trace:
I20260709 12:17:56.891899 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:56.751787 21735 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.895038 18447 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:17:56.895483 18447 pending_rounds.cc:85] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:17:56.895748 18447 pending_rounds.cc:107] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:56.896067 18447 raft_consensus.cc:2889] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:56.863336 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:56.896653 19044 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 5.
I20260709 12:17:56.862946 18446 rpcz_store.cc:276] 0709 12:17:54.747834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.791887 (+2044053us) service_pool.cc:224] Handling call
0709 12:17:56.792271 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.792451 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.792474 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.792514 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:56.792547 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.792610 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.792718 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.897104 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:56.898010 18446 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:56.898139 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:56.898569 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 732) took 1887 ms. Trace:
I20260709 12:17:56.866736 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:56.868918 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.869292 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1820) took 2206 ms. Trace:
I20260709 12:17:56.851270 21798 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:56.881578 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:56.888566 21347 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:56.901330 18449 rpcz_store.cc:276] 0709 12:17:54.661947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.514326 (+1852379us) service_pool.cc:224] Handling call
0709 12:17:56.514739 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.515011 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:56.686258 (+171247us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.868127 (+181869us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.868196 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:56.868262 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.868347 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.868495 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14706,"prepare.queue_time_us":239813,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":2575,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":11967,"thread_start_us":349682,"threads_started":2}
I20260709 12:17:56.892346 18445 rpcz_store.cc:276] 0709 12:17:54.632784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.299034 (+1666250us) service_pool.cc:224] Handling call
0709 12:17:56.342291 (+ 43257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.362631 (+ 20340us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:56.603023 (+240392us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.862765 (+259742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:56.862816 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:56.862865 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.862924 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:56.863034 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":367369,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":617,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":10146,"thread_start_us":498765,"threads_started":2}
I20260709 12:17:56.758838 21504 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.898933 18446 rpcz_store.cc:276] 0709 12:17:55.011613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.897506 (+1885893us) service_pool.cc:224] Handling call
0709 12:17:56.897751 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.898349 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.898511 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.879997 21814 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:56.905756 19041 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:56.847687 21696 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.908605 19042 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:17:56.908835 18445 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:17:56.918651 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1825) took 2104 ms. Trace:
I20260709 12:17:56.909096 21687 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.864682 21531 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.922178 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:17:56.922715 21531 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.923552 21824 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:56.923777 21568 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.923970 21824 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.922755 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:56.903352 21680 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.909271 21776 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.847376 21799 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.933130 21799 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.892812 21673 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.910027 21798 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.910568 21777 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.910910 21735 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:17:56.896736 21648 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:56.889253 21526 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.858682 21791 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.910602 21347 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:56.913394 21819 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:56.892580 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:56.870275 21805 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:56.855768 21709 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.909375 21563 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.925442 19036 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:56.926090 21824 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:56.926666 21504 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.926518 21818 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.925012 21577 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:56.928454 21696 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.929154 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:56.861806 21708 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.914490 21417 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.908878 21669 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.945473 21791 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.953751 21709 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.955122 21667 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:56.955713 21708 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.957227 21667 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.939978 21347 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:56.948482 21577 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.948642 21818 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.947710 21805 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.964876 21577 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:56.956952 21648 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.965919 21648 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:56.958811 21780 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.309s	user 0.004s	sys 0.029s
I20260709 12:17:56.967432 21682 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.947824 21677 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:56.962365 21764 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.964154 21678 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:56.964939 21818 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.918470 21814 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:56.969822 21790 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:56.959347 21765 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.944s	user 0.009s	sys 0.019s
I20260709 12:17:56.970491 21764 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.973846 21814 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.978152 21814 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.985715 21824 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:56.988974 21824 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:56.988476 21818 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.024677 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:57.026352 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 572) took 1891 ms. Trace:
W20260709 12:17:57.026942 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 128731us
I20260709 12:17:57.028820 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:57.032549 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:57.027041 18291 rpcz_store.cc:276] 0709 12:17:55.134842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.031959 (+897117us) service_pool.cc:224] Handling call
0709 12:17:56.035614 (+  3655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.035852 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:56.684532 (+648680us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.025261 (+340729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.025408 (+   147us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:57.025545 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.025704 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.026013 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":600443,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3098,"thread_start_us":919344,"threads_started":2,"wal-append.queue_time_us":265610}
W20260709 12:17:57.035948 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 154314us
W20260709 12:17:57.036370 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.912957 18370 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:57.040613 18368 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:17:57.041356 18368 pending_rounds.cc:85] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:17:57.041765 18368 pending_rounds.cc:107] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:17:57.042205 18368 raft_consensus.cc:2889] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.044245 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1836) took 3041 ms. Trace:
I20260709 12:17:57.044781 18371 rpcz_store.cc:276] 0709 12:17:54.002183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.159862 (+157679us) service_pool.cc:224] Handling call
0709 12:17:54.160211 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.160562 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.160590 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.161083 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:17:56.376577 (+2215494us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b18004be260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:17:56.643480 (+266903us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:56.643522 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.643789 (+   267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.043465 (+399676us) raft_consensus.cc:1685] finished
0709 12:17:57.043515 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.043983 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1187,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":12182,"spinlock_wait_cycles":3859328,"thread_start_us":266686,"threads_started":1,"wal-append.queue_time_us":2735361}
I20260709 12:17:57.048926 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 717) took 2518 ms. Trace:
I20260709 12:17:57.049319 18441 rpcz_store.cc:276] 0709 12:17:54.529889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.060202 (+530313us) service_pool.cc:224] Handling call
0709 12:17:55.060506 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:55.060872 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:55.060910 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:55.063257 (+  2347us) log.cc:844] Serialized 26 byte log entry
0709 12:17:55.298170 (+234913us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:55.298209 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.298268 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:56.958243 (+1659975us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:17:57.048301 (+ 90058us) raft_consensus.cc:1685] finished
0709 12:17:57.048352 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.048700 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":978,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4586,"spinlock_wait_cycles":89468800,"thread_start_us":197916,"threads_started":1,"wal-append.queue_time_us":550002}
I20260709 12:17:57.051862 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.054128 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.054867 18441 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:56.919188 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2241) took 1844 ms. Trace:
I20260709 12:17:57.060083 18444 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:17:57.059666 18445 rpcz_store.cc:276] 0709 12:17:55.075053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.908326 (+1833273us) service_pool.cc:224] Handling call
0709 12:17:56.908632 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.918961 (+ 10329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.919137 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:56.919029 18447 rpcz_store.cc:276] 0709 12:17:54.814058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.871520 (+2057462us) service_pool.cc:224] Handling call
0709 12:17:56.894726 (+ 23206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.918419 (+ 23693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.918601 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.061347 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.061823 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.062671 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.063473 18445 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:17:57.064098 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1826) took 1924 ms. Trace:
I20260709 12:17:57.064489 18447 rpcz_store.cc:276] 0709 12:17:55.139702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.062860 (+1923158us) service_pool.cc:224] Handling call
0709 12:17:57.063414 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.063778 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.063802 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.063862 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.063894 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.063941 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.064034 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.065343 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1827) took 1921 ms. Trace:
I20260709 12:17:57.065759 18445 rpcz_store.cc:276] 0709 12:17:55.143683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.064096 (+1920413us) service_pool.cc:224] Handling call
0709 12:17:57.064489 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.064967 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.064995 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.065064 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:57.065137 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.065190 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.065291 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:57.066797 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1831) took 1455 ms. Trace:
I20260709 12:17:57.067195 18447 rpcz_store.cc:276] 0709 12:17:55.611134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.065554 (+1454420us) service_pool.cc:224] Handling call
0709 12:17:57.065993 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.066439 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.066469 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.066536 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.066577 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.066627 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.066726 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:57.075778 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.077248 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.078156 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:57.079308 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:57.084265 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 197656us
I20260709 12:17:57.086776 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.087560 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:57.089093 19039 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:17:56.935946 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:57.085291 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.091651 21347 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.094411 21347 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:57.094658 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.084688 19006 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.750s (SENT) [suppressed 16 similar messages]
I20260709 12:17:56.962684 21417 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:57.100445 21417 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.107316 21504 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.108311 21504 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.034881 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:57.113155 21568 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.114148 21568 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:57.114943 21568 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:57.115942 21568 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:57.116317 21568 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.116693 21568 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.122371 21631 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.122954 21667 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:57.123306 21631 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.124945 21667 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:57.125274 21662 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:57.125893 21667 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:57.126368 21667 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:17:57.128064 21655 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:57.127928 21667 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:57.129109 21662 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:17:57.129041 21655 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:57.137025 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1753) took 1531 ms. Trace:
I20260709 12:17:57.150868 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 652) took 2717 ms. Trace:
I20260709 12:17:57.151429 18369 rpcz_store.cc:276] 0709 12:17:54.433549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.648012 (+214463us) service_pool.cc:224] Handling call
0709 12:17:54.657203 (+  9191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.024249 (+367046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.150239 (+2125990us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400168ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:57.150664 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842849536}
I20260709 12:17:57.088415 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:17:57.211695 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 127239us
I20260709 12:17:57.137518 18293 rpcz_store.cc:276] 0709 12:17:55.605727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.166159 (+560432us) service_pool.cc:224] Handling call
0709 12:17:56.166496 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:56.300553 (+134057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.300584 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:56.315173 (+ 14589us) log.cc:844] Serialized 52 byte log entry
0709 12:17:56.502806 (+187633us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:56.836742 (+333936us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.838312 (+  1570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.136274 (+297962us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400001754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:57.136350 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.136426 (+    76us) raft_consensus.cc:1685] finished
0709 12:17:57.136532 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.136816 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71,"prepare.queue_time_us":167891,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":42623,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":25952,"spinlock_wait_cycles":307088640,"thread_start_us":471744,"threads_started":2,"wal-append.queue_time_us":334738}
I20260709 12:17:57.229575 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.229920 21631 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:57.234612 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.237922 21667 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.242101 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.243906 18448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2204) took 4708 ms (client timeout 2115 ms). Trace:
W20260709 12:17:57.244532 18448 rpcz_store.cc:269] 0709 12:17:52.535706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.574673 (+ 38967us) service_pool.cc:224] Handling call
0709 12:17:57.243375 (+4668702us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400376254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:57.243622 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2892040960}
I20260709 12:17:57.246220 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.208283 21417 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:57.248037 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.255544 21504 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:57.256174 21417 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.262487 21663 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.263384 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.262969 21805 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.259369 21791 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.291563 18447 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
W20260709 12:17:57.294668 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.299296 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2246) took 2221 ms. Trace:
I20260709 12:17:57.301012 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.311936 18448 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:57.311373 18449 rpcz_store.cc:276] 0709 12:17:55.077957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.916910 (+1838953us) service_pool.cc:224] Handling call
0709 12:17:56.936897 (+ 19987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.937431 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.937488 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.069656 (+132168us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.220111 (+150455us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:57.227463 (+  7352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.235824 (+  8361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.298851 (+ 63027us) raft_consensus.cc:1685] finished
0709 12:17:57.298880 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.299069 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1943,"thread_start_us":149913,"threads_started":1,"wal-append.queue_time_us":226578}
W20260709 12:17:57.301682 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.301883 21754 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.302785 21764 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.321528 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.322186 18449 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.307476 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.312955 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.318213 21754 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.330101 21727 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:57.331875 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:57.329170 18448 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.335047 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 44557us
I20260709 12:17:57.338464 21417 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.360625 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:57.360967 21417 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:57.361538 18369 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:57.362094 21791 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.338788 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 638) took 4457 ms. Trace:
I20260709 12:17:57.339578 19044 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:57.339046 19042 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:57.339890 21818 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:57.344929 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:57.345515 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:57.295730 21805 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.347801 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 731) took 2533 ms. Trace:
I20260709 12:17:57.348809 19039 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:17:57.351585 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:57.332517 21754 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.356714 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:57.357586 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:57.362131 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:57.364737 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.345321 21568 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:57.778020 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.715847 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 204581us
I20260709 12:17:57.463209 18449 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:57.364567 18368 rpcz_store.cc:276] 0709 12:17:52.880942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.881845 (+   903us) service_pool.cc:224] Handling call
0709 12:17:52.882217 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.043103 (+4160886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.338414 (+295311us) spinlock_profiling.cc:243] Waited 4.16 s on lock 0x7b54003f2d54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:57.338691 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9560218496}
W20260709 12:17:57.501431 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 466775us
W20260709 12:17:57.682420 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.840240 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2247) took 2238 ms. Trace:
W20260709 12:17:57.766444 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 188581us
I20260709 12:17:57.367118 18444 rpcz_store.cc:276] 0709 12:17:54.814822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.900229 (+2085407us) service_pool.cc:224] Handling call
0709 12:17:56.909160 (+  8931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.060706 (+151546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.347504 (+286798us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000d9654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:57.347748 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345330944}
W20260709 12:17:57.768666 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.930378 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 449492us
W20260709 12:17:57.932528 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.925326 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.582389 18369 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:17:57.920476 18443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.022s	sys 0.010s
W20260709 12:17:57.936965 18443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.022s	sys 0.010s
I20260709 12:17:57.523731 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:57.332381 21635 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:57.716558 18441 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.003s
I20260709 12:17:57.375566 19043 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:17:57.661248 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1749) took 2768 ms. Trace:
I20260709 12:17:57.391683 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:17:57.770969 21837 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.523s	user 0.006s	sys 0.000s
W20260709 12:17:57.640527 21577 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.012s
I20260709 12:17:57.945413 18441 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
W20260709 12:17:57.622337 21774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.958s	user 0.019s	sys 0.021s
I20260709 12:17:57.386267 21597 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:17:57.377425 21771 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 2.365s	user 0.009s	sys 0.027s
I20260709 12:17:57.953315 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:57.422669 21791 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.958171 21597 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:57.968116 21597 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:57.969537 21597 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.964445 21676 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.971489 21676 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:57.971980 21676 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.974877 18198 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:57.964882 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 618737us
I20260709 12:17:57.990342 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:17:57.993467 21676 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.001170 21560 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.001965 21560 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.002889 21676 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.004634 21560 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:57.660753 21550 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.927803 21417 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:57.581313 21829 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.422643 21735 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.928133 18446 rpcz_store.cc:276] 0709 12:17:55.091578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.915864 (+1824286us) service_pool.cc:224] Handling call
0709 12:17:56.916157 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.916592 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.916642 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:56.917102 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.234123 (+317021us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.240422 (+  6299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.241144 (+   722us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.308852 (+ 67708us) raft_consensus.cc:1685] finished
0709 12:17:57.308868 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.329942 (+ 21074us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":167717,"threads_started":1,"wal-append.queue_time_us":383458}
I20260709 12:17:58.010361 18370 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
W20260709 12:17:58.018728 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 72640.7us
W20260709 12:17:57.957628 21774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.302s	user 0.019s	sys 0.040s
W20260709 12:17:58.027908 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 439880us
I20260709 12:17:58.044828 18447 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.959762 21754 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.424469 21662 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.588s
W20260709 12:17:58.047199 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 85372.4us
I20260709 12:17:58.049539 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:57.945497 18299 rpcz_store.cc:276] 0709 12:17:54.893043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.035332 (+1142289us) service_pool.cc:224] Handling call
0709 12:17:56.035686 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:56.566287 (+530601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.566326 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:56.593590 (+ 27264us) log.cc:844] Serialized 52 byte log entry
0709 12:17:56.838013 (+244423us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:56.856064 (+ 18051us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.961432 (+105368us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b18004d6860. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:17:57.362855 (+401423us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b18004d6860. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:17:57.363201 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.607786 (+244585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.623102 (+ 15316us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.659203 (+ 36101us) raft_consensus.cc:1685] finished
0709 12:17:57.659222 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.661058 (+  1836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":594653,"prepare.queue_time_us":524735,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":420140,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":7389,"spinlock_wait_cycles":20358400,"thread_start_us":701396,"threads_started":2,"wal-append.queue_time_us":287400}
W20260709 12:17:58.059819 18448 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.000s
I20260709 12:17:58.060235 21642 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:58.066941 21818 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.067413 21818 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:58.067783 21818 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.070967 18371 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.057315 18366 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:58.078889 18448 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:58.066979 21642 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:58.080597 21648 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.114s	user 0.020s	sys 0.001s
I20260709 12:17:58.082945 21648 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:17:58.087273 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2238) took 3631 ms. Trace:
I20260709 12:17:58.087905 18440 rpcz_store.cc:276] 0709 12:17:54.455967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.752713 (+296746us) service_pool.cc:224] Handling call
0709 12:17:54.758567 (+  5854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.810316 (+ 51749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.810349 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.820162 (+  9813us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.730914 (+2910752us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b18003bd760. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:17:58.011764 (+280850us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:58.024287 (+ 12523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.051665 (+ 27378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.086659 (+ 34994us) raft_consensus.cc:1685] finished
0709 12:17:58.086686 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.086973 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56857,"raft.queue_time_us":51401,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8176,"spinlock_wait_cycles":6697216,"thread_start_us":133466,"threads_started":1,"wal-append.queue_time_us":3256314}
I20260709 12:17:58.092504 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:58.093883 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1838) took 1373 ms. Trace:
I20260709 12:17:58.096594 18441 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:58.093308 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.092319 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2249) took 1610 ms. Trace:
I20260709 12:17:58.038509 21791 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:17:58.099071 19043 raft_consensus.cc:2410] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:17:58.099223 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2250) took 1572 ms. Trace:
I20260709 12:17:58.100610 18375 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:58.098889 18444 rpcz_store.cc:276] 0709 12:17:56.720820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.011142 (+1290322us) service_pool.cc:224] Handling call
0709 12:17:58.093131 (+ 81989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.093429 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.093476 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.093554 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:58.093604 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.093674 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.093794 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.102000 18441 rpcz_store.cc:276] 0709 12:17:56.526527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.034531 (+1508004us) service_pool.cc:224] Handling call
0709 12:17:58.067981 (+ 33450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.098991 (+ 31010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.099175 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:58.105939 21727 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.002s
I20260709 12:17:58.106220 18198 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:57.378783 21787 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.109421 21642 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:58.110019 21642 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.110659 21642 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.113708 21642 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.053248 21635 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.110020 21787 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:58.116127 18415 connection.cc:256] RPC call timeout handler was delayed by -1.2819814739998492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 12:17:58.116614 18415 connection.cc:256] RPC call timeout handler was delayed by -1.1469814739998583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 12:17:58.116931 18415 connection.cc:256] RPC call timeout handler was delayed by -1.0199299119999523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 12:17:58.117262 18415 connection.cc:256] RPC call timeout handler was delayed by -1.0009340550000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260709 12:17:58.024612 21577 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.124114 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.122994 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:57.380101 21802 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.100390 21560 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.130710 21798 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.099370 18443 rpcz_store.cc:276] 0709 12:17:56.482257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.339860 (+857603us) service_pool.cc:224] Handling call
0709 12:17:57.340282 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.340864 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.340897 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.341479 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.937448 (+595969us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:57.937516 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.937604 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.092006 (+154402us) raft_consensus.cc:1685] finished
0709 12:17:58.092032 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.092209 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5960,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":123185,"thread_start_us":595532,"threads_started":1,"wal-append.queue_time_us":739905}
I20260709 12:17:58.138914 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:17:58.133073 21667 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.003s
W20260709 12:17:58.141695 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.073559 21754 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.119742 21662 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.142689 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:17:58.139168 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.147616 21798 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.150147 21798 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.136620 21631 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:17:58.151985 21560 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.154155 21568 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.146142 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.156344 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:58.156509 19035 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:17:58.156785 18443 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:58.156792 19042 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:58.152628 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:58.092844 21735 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.134568 21417 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.165824 18447 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:58.101015 21829 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.168045 18449 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:58.167392 19043 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:58.168228 18446 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:17:58.169070 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.157437 21729 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:58.173008 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.173779 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:17:58.164146 21642 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:17:58.179548 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.182278 18441 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.181751 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.173976 21798 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.188247 19035 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:58.190412 18447 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:17:58.187345 21527 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.184410 21635 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.192682 21799 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:58.206382 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53876.5us
I20260709 12:17:58.188910 21727 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.189134 21577 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.211447 21564 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:58.206257 21560 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.217767 21560 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:58.221153 18299 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:17:58.221881 18442 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:58.220424 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:58.206336 21805 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.189270 21667 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.200327 18443 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:17:58.217160 21798 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.225687 21841 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.237867 21841 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.239904 21841 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.220865 21398 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.240650 21398 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.241827 21829 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.242184 19035 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:17:58.242607 21398 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.218617 21799 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.185714 21662 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.228071 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1839) took 1460 ms. Trace:
I20260709 12:17:58.246019 18446 rpcz_store.cc:276] 0709 12:17:56.768187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.047052 (+1278865us) service_pool.cc:224] Handling call
0709 12:17:58.166934 (+119882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.171392 (+  4458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.227902 (+ 56510us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b54001a6650. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:58.228009 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98730752}
I20260709 12:17:58.232517 21564 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:58.229140 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:58.230499 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1836) took 1510 ms. Trace:
I20260709 12:17:58.191741 21417 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.247430 18449 rpcz_store.cc:276] 0709 12:17:56.719749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.966100 (+1246351us) service_pool.cc:224] Handling call
0709 12:17:57.966653 (+   553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.168580 (+201927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.230280 (+ 61700us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54001a5754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:17:58.230438 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461797632}
I20260709 12:17:58.198022 21774 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:17:58.234980 21805 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.218472 18198 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 4, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:58.236248 21560 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.232645 21635 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.197227 21527 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:58.237145 21568 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:58.235674 21754 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.236217 21735 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:58.255065 21774 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.255152 21564 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.256361 18447 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:17:58.269177 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.219342 21727 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.259222 21799 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:58.266412 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.257738 21791 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:58.270751 18369 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:17:58.253868 21560 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:58.258256 21635 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:58.264349 18198 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 4, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.282444 21527 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:58.283464 21635 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.285454 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.286113 19035 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:17:58.279790 21560 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.261845 21841 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:58.286118 21735 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.290728 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.297991 21662 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.298615 18198 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.300591 18198 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:58.275414 21805 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:58.303788 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 74773.3us
I20260709 12:17:58.302312 18198 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 5, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:17:58.279876 21568 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.259379 21667 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.324133 18441 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:58.343690 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2264) took 1003 ms. Trace:
I20260709 12:17:58.344251 18441 rpcz_store.cc:276] 0709 12:17:57.340440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.177222 (+836782us) service_pool.cc:224] Handling call
0709 12:17:58.178785 (+  1563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.343267 (+164482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.343607 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.305991 21791 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:17:58.369361 21774 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.313226 21841 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.305167 21735 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.381304 21648 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.363755 21754 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:17:58.398362 21798 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.363567 21722 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 12:17:58.400101 21727 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.401506 21560 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:17:58.403362 21845 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:17:58.405373 21805 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.407306 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.408880 18366 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:17:58.409615 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:58.413048 21802 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:58.443276 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.417985 18371 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:58.385723 21791 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.453519 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 672) took 1087 ms. Trace:
I20260709 12:17:58.453716 18373 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:58.454381 18371 rpcz_store.cc:276] 0709 12:17:57.357136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.357284 (+   148us) service_pool.cc:224] Handling call
0709 12:17:58.444342 (+1087058us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:58.454721 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.457508 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.407063 21818 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.447531 21798 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.457159 21417 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.456120 21676 raft_consensus.cc:2764] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:58.419379 21648 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.472632 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:58.477952 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 487754us
I20260709 12:17:58.482805 18369 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:58.486403 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.471959 21631 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.493242 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.495071 19039 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 12:17:58.498808 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 261886us
W20260709 12:17:58.498879 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.505926 21568 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.506697 19037 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:58.507869 18370 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:17:58.507722 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 98268.2us
I20260709 12:17:58.509847 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:17:58.516882 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1650246359999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
I20260709 12:17:58.514387 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.523010 19041 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:58.483085 21798 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.526332 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.529057 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.526072 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:58.532924 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.534340 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.534598 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.534961 19038 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:17:58.535463 18300 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.496385 21631 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.537467 21631 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.565919 18299 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 12:17:58.534581 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.567448 18298 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.524314 21667 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.528240 21662 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.513240 21550 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:17:58.584385 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.593400 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.595887 18373 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.599792 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1840) took 1829 ms. Trace:
I20260709 12:17:58.601766 18448 rpcz_store.cc:276] 0709 12:17:56.770598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.103952 (+1333354us) service_pool.cc:224] Handling call
0709 12:17:58.129173 (+ 25221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.139054 (+  9881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.139089 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.147363 (+  8274us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.465222 (+317859us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:58.465279 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.465373 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.599544 (+134171us) raft_consensus.cc:1685] finished
0709 12:17:58.599570 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.599714 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7484,"raft.queue_time_us":5366,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":11101,"thread_start_us":317404,"threads_started":1,"wal-append.queue_time_us":375888}
W20260709 12:17:58.609023 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:58.610509 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.615602 18371 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:58.514725 21799 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:17:58.568661 21398 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:17:58.626569 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 106725us
I20260709 12:17:58.568450 21560 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.621902 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.625590 18366 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:58.632623 18299 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:17:58.629142 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 100110us
I20260709 12:17:58.634277 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.637171 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:58.645303 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116322us
I20260709 12:17:58.669306 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.671504 18300 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:58.674556 18369 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:58.675107 18291 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.670691 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:58.679118 19040 raft_consensus.cc:2410] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 12:17:58.680307 18447 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:58.681617 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.682363 18369 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:58.638278 21841 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.700011 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 837) took 1160 ms. Trace:
I20260709 12:17:58.646488 21791 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.637862 21577 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.710152 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.710199 18299 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:17:58.629931 21662 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.709375 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.638419 21597 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:17:58.678970 21818 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:58.690449 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 737) took 1459 ms. Trace:
W20260709 12:17:58.725993 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.681797 21398 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:58.733546 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.701998 19036 rpcz_store.cc:276] 0709 12:17:57.349893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.350029 (+   136us) service_pool.cc:224] Handling call
0709 12:17:57.350453 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.422956 (+ 72503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.422994 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.608038 (+185044us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.080609 (+472571us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:58.080657 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.080718 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.510484 (+429766us) raft_consensus.cc:1685] finished
0709 12:17:58.510515 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.510679 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77104,"raft.queue_time_us":73596,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":224,"thread_start_us":434169,"threads_started":1,"wal-append.queue_time_us":517780}
I20260709 12:17:58.725746 18440 rpcz_store.cc:276] 0709 12:17:57.231100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.090521 (+859421us) service_pool.cc:224] Handling call
0709 12:17:58.090960 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.091380 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:58.448680 (+357300us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.689275 (+240595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.689384 (+   109us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:58.689459 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.690091 (+   632us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.690369 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":405335,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1769,"thread_start_us":568949,"threads_started":3}
I20260709 12:17:58.741164 18447 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:58.739849 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 740) took 1507 ms. Trace:
I20260709 12:17:58.742614 18444 rpcz_store.cc:276] 0709 12:17:57.232487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.104394 (+871907us) service_pool.cc:224] Handling call
0709 12:17:58.133125 (+ 28731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.144714 (+ 11589us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:58.482687 (+337973us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.739533 (+256846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.739600 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:58.739645 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.739706 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.739798 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":360617,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"thread_start_us":537141,"threads_started":3}
W20260709 12:17:58.726935 21787 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 12:17:58.747416 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.740675 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.749857 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.750419 18448 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:58.751451 18300 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:58.638010 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:17:58.727576 21855 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:58.752303 19043 raft_consensus.cc:606] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:58.738896 21667 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.760739 18445 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:17:58.759835 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.764175 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.787662 19043 raft_consensus.cc:614] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:58.741290 21791 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.794387 18295 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:17:58.802109 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.807533 18447 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:58.787726 18293 raft_consensus.cc:2376] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7. Re-sending same reply.
I20260709 12:17:58.764158 21798 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.793267 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:17:58.811517 18370 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.794641 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.811082 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.760190 21847 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.819766 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:58.810649 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.826609 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.827096 18440 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:58.828085 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.748199 21577 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.760881 21648 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.814252 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.792249 21642 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.839493 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.840569 18367 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:17:58.827656 21791 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.843271 21847 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.844683 21791 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:58.845345 21847 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:58.847026 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.846285 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.797061 21667 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:17:58.861054 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59809.9us
I20260709 12:17:58.865993 18295 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:17:58.838682 21597 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.867959 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.871104 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:58.874182 18445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2262) took 1548 ms (client timeout 1575 ms). Trace:
I20260709 12:17:58.855196 21847 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.876336 18296 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
W20260709 12:17:58.876468 18445 rpcz_store.cc:269] 0709 12:17:57.325691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.332559 (+  6868us) service_pool.cc:224] Handling call
0709 12:17:58.863395 (+1530836us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001a5254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:58.863538 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175620864}
I20260709 12:17:58.856709 21848 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.884829 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.885370 18293 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:58.882176 18295 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:17:58.857220 21855 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:17:58.893232 21866 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:58.890621 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:58.897190 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:58.875025 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.899371 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.900815 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:58.833505 21856 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.237s
I20260709 12:17:58.879065 21597 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:17:58.915361 21564 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.915823 21564 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.916198 21564 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.916353 21597 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.917492 21597 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.917956 21856 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:17:58.852144 21642 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.918378 21856 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:58.919314 21856 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.922014 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.874985 21864 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.929100 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.933866 18291 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:58.894070 21862 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:58.936259 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:58.876284 21727 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:17:58.920621 21648 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:58.942073 18371 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:58.951632 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:58.933741 21856 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:58.965065 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:58.967063 21735 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:58.962422 18444 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:17:59.010623 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:59.011541 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60022.1us
W20260709 12:17:59.012713 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.013157 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44435.9us
I20260709 12:17:59.009987 18292 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:58.945930 21648 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.009442 21859 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.027798 21859 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:58.898471 21848 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.924327 21847 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:58.875712 21398 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:59.039281 18298 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:17:59.040908 18299 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:17:58.925194 21855 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:59.043638 21855 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:59.045615 21847 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:59.069172 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 319807us
I20260709 12:17:59.032966 21735 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:59.032202 21648 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:59.102241 19006 leader_election.cc:336] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.750s (SENT)
W20260709 12:17:59.126348 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50529.3us
I20260709 12:17:59.036619 21662 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:59.128492 18298 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:17:59.129580 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.130712 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:59.132601 21871 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:17:59.133780 21642 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:59.141498 18293 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:17:59.146804 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:17:59.127254 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:58.943109 21864 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.151913 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:17:59.125077 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.000s	sys 0.012s
I20260709 12:17:59.176530 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:17:59.177994 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.000s	sys 0.012s
I20260709 12:17:59.178186 18293 raft_consensus.cc:2376] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
W20260709 12:17:59.178500 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 640065us
W20260709 12:17:59.136684 21865 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 12:17:59.112308 21735 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.194752 21848 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:59.195151 21735 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:59.201284 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49473.3us
W20260709 12:17:59.207791 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 105610us
W20260709 12:17:59.209623 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.175663 21729 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:59.205978 21870 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.391s	user 0.000s	sys 0.001s
I20260709 12:17:59.276149 21564 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.277431 21644 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.609026 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.371397 18373 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.000s
W20260709 12:17:59.609845 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.288749 21577 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:17:59.545998 21635 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.260s	user 0.013s	sys 0.000s
I20260709 12:17:59.335417 21631 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.612082 21631 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.611827 21635 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:59.612789 21631 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:59.524264 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.003s	sys 0.029s
I20260709 12:17:59.613181 18442 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:59.613448 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.004s	sys 0.034s
I20260709 12:17:59.609808 18373 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:59.614563 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:59.615082 21644 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.619608 18368 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:59.377826 18339 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.008s	sys 0.005s
I20260709 12:17:59.620810 21635 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:59.620994 18339 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.008s	sys 0.010s
I20260709 12:17:59.602051 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.621832 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:17:59.622148 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 592555us
I20260709 12:17:59.622745 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.623541 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:59.623931 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.616966 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.618463 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.619505 19036 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:17:59.362926 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 125734us
W20260709 12:17:59.621461 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 950465us
I20260709 12:17:59.624161 18291 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.616250 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:17:59.624851 18370 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.810s	user 0.014s	sys 0.019s
I20260709 12:17:59.652904 18370 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:17:59.653081 18292 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:59.634356 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:17:59.654089 18296 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:17:59.611292 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.654942 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.656284 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.658700 18292 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:17:59.654382 21885 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.654596 21635 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:59.610018 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:59.619503 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1872) took 1079 ms. Trace:
W20260709 12:17:59.603039 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.632352 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:17:59.247596 21727 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:59.310618 21802 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.854s	user 0.008s	sys 0.000s
I20260709 12:17:59.663944 18373 rpcz_store.cc:276] 0709 12:17:58.540198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.540336 (+   138us) service_pool.cc:224] Handling call
0709 12:17:59.619436 (+1079100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:59.664674 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.666522 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.666944 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 686) took 1013 ms. Trace:
W20260709 12:17:59.315498 21868 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 12:17:59.280671 21729 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.603118 21882 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.663920 18294 raft_consensus.cc:2376] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12. Re-sending same reply.
I20260709 12:17:59.669252 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.670229 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.670251 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:59.606881 21642 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:17:59.692251 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:17:59.692819 18443 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.692976 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.693682 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.694393 18440 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:59.673156 18295 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:17:59.667505 21635 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.675637 21859 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:59.675881 21892 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:59.676749 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 44490us
I20260709 12:17:59.652472 21864 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:59.678962 21722 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.679613 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:17:59.700641 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.701128 18366 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:17:59.701416 18447 raft_consensus.cc:2410] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 12:17:59.679554 21631 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.681978 19035 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:59.299896 21667 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:17:59.603576 21797 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.687328 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.688570 21564 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:59.691388 18449 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:17:59.658532 21735 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.698195 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.694706 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.711633 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.692376 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:59.705785 18291 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:17:59.736879 18445 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:59.737075 18296 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:17:59.706694 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.697089 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:59.707381 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:17:59.689491 21729 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.708196 18293 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:17:59.696986 21642 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:17:59.740023 18446 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:17:59.741003 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.741079 18448 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:17:59.741263 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 109961us
I20260709 12:17:59.741629 18291 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:17:59.742328 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:17:59.669461 21885 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.699483 21635 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:59.704105 21648 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.528s	user 0.005s	sys 0.005s
I20260709 12:17:59.745514 18291 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:59.746954 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47769.1us
I20260709 12:17:59.747182 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:17:59.697336 21527 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.752954 18440 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:59.701184 21798 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:17:59.738374 21864 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.747761 21550 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:17:59.735028 21797 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.738888 21872 raft_consensus.cc:2764] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:17:59.759039 18295 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:59.766819 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2169) took 1010 ms. Trace:
I20260709 12:17:59.701047 21894 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:17:59.762569 21642 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:59.758818 18292 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:59.697443 21882 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.771569 18300 rpcz_store.cc:276] 0709 12:17:58.761423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.859872 (+ 98449us) service_pool.cc:224] Handling call
0709 12:17:58.883070 (+ 23198us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:58.944971 (+ 61901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.945000 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:59.013731 (+ 68731us) log.cc:844] Serialized 52 byte log entry
0709 12:17:59.219457 (+205726us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:59.219519 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.219606 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.766616 (+547010us) raft_consensus.cc:1685] finished
0709 12:17:59.766636 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.766767 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1623,"raft.queue_time_us":7334,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":63354,"thread_start_us":187641,"threads_started":1,"wal-append.queue_time_us":662371}
I20260709 12:17:59.706349 21722 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.748163 21648 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.775754 21900 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:17:59.817621 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.767022 21735 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:17:59.835552 18443 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:17:59.764310 21797 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:17:59.840395 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 88514.7us
I20260709 12:17:59.845786 21631 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:17:59.747938 21662 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
W20260709 12:17:59.855633 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 107969us
I20260709 12:17:59.762104 21550 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:17:59.755203 21417 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:17:59.828034 21900 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.828328 21564 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:17:59.773949 21864 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:17:59.900723 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1727) took 1089 ms. Trace:
I20260709 12:17:59.759732 21896 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 12:17:59.910957 18368 rpcz_store.cc:276] 0709 12:17:58.811663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.923871 (+112208us) service_pool.cc:224] Handling call
0709 12:17:59.619436 (+695565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.620069 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.900461 (+280392us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000c9ed0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:17:59.900649 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588019584}
I20260709 12:17:59.915622 21417 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:59.916455 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:17:59.917150 21892 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:17:59.928083 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 181839us
W20260709 12:17:59.958045 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 212863us
W20260709 12:17:59.957978 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 348397us
I20260709 12:17:59.914424 21662 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:59.898363 21667 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:17:59.976169 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 316830us
I20260709 12:17:59.908967 21900 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:00.044080 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 300399us
I20260709 12:17:59.987983 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 682) took 1319 ms. Trace:
I20260709 12:17:59.908679 21577 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:17:59.967622 21662 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:17:59.927629 21564 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:17:59.965377 21417 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:00.126072 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:17:59.964690 21841 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.162s
I20260709 12:18:00.049244 18443 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:00.191172 21916 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:00.204823 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 154441us
I20260709 12:18:00.205098 21916 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:00.065430 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 102752us
W20260709 12:18:00.203701 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 221657us
I20260709 12:18:00.207494 21916 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.209741 18299 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:00.210549 21916 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:00.211972 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:00.125221 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:00.068076 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 94848.2us
I20260709 12:17:59.668639 18367 rpcz_store.cc:276] 0709 12:17:58.653900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.859713 (+205813us) service_pool.cc:224] Handling call
0709 12:17:58.878879 (+ 19166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.890724 (+ 11845us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:17:58.933414 (+ 42690us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.666628 (+733214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.666681 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:17:59.666729 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.666790 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.666892 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":557153,"prepare.queue_time_us":12843,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":554510,"thread_start_us":175040,"threads_started":1,"wal-append.queue_time_us":673931}
W20260709 12:18:00.181538 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 265156us
W20260709 12:17:59.983249 21856 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.972s	user 0.011s	sys 0.000s
W20260709 12:18:00.228175 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.117022 18297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.005s	sys 0.007s
W20260709 12:18:00.215458 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.214192 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 89086.4us
W20260709 12:18:00.210609 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.033301 21667 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.390839 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:00.252079 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 286721us
I20260709 12:17:59.991627 21642 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:00.395917 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:00.396914 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1732) took 1369 ms. Trace:
I20260709 12:18:00.397769 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:00.398837 18372 raft_consensus.cc:2410] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:18:00.392725 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:00.399637 18412 connection.cc:256] RPC call timeout handler was delayed by -1.8872988890000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 12:18:00.388435 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:00.399969 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:00.397342 18371 rpcz_store.cc:276] 0709 12:17:59.027348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.040614 (+ 13266us) service_pool.cc:224] Handling call
0709 12:18:00.395745 (+1355131us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5800083a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:00.396846 (+  1101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35738880}
I20260709 12:18:00.400337 18446 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:00.274549 21903 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:18:00.402374 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:00.403183 19042 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:00.319851 18449 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:00.273806 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:00.406634 18368 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.408437 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:00.409796 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1737) took 1380 ms. Trace:
W20260709 12:18:00.400058 18412 connection.cc:256] RPC call timeout handler was delayed by -1.8557988889999706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 12:18:00.410936 18412 connection.cc:256] RPC call timeout handler was delayed by -1.834198889000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 12:18:00.411414 18412 connection.cc:256] RPC call timeout handler was delayed by -1.6865988890001518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
W20260709 12:18:00.411914 18412 connection.cc:256] RPC call timeout handler was delayed by -1.6831913099999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
W20260709 12:18:00.412294 18412 connection.cc:256] RPC call timeout handler was delayed by -1.6082988890000252s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
W20260709 12:18:00.412653 18412 connection.cc:256] RPC call timeout handler was delayed by -1.4155519989999448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
W20260709 12:18:00.413038 18412 connection.cc:256] RPC call timeout handler was delayed by -1.2510933399998976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 12:18:00.410712 18372 rpcz_store.cc:276] 0709 12:17:59.029058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.196406 (+167348us) service_pool.cc:224] Handling call
0709 12:18:00.397599 (+1201193us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b5800083a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:00.409742 (+ 12143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10774784}
I20260709 12:18:00.419691 19041 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:00.420421 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.421248 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:00.409418 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1734) took 1381 ms. Trace:
I20260709 12:18:00.262557 18298 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:18:00.234615 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:00.269618 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:00.229964 18297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.010s	sys 0.014s
I20260709 12:18:00.426594 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:00.423220 18375 rpcz_store.cc:276] 0709 12:17:59.028102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.046030 (+ 17928us) service_pool.cc:224] Handling call
0709 12:18:00.392539 (+1346509us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b5800083a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:00.409369 (+ 16830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52600704}
I20260709 12:18:00.427556 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:00.428335 18371 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:00.430265 19038 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:00.420341 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:00.430364 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.422541 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:00.230409 21735 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:00.095698 21859 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.116811 21885 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:00.435726 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44948.6us
I20260709 12:18:00.125622 21892 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:00.262568 21648 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:00.084337 21848 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.352s
I20260709 12:18:00.446671 19039 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:00.121182 21864 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:00.395867 21642 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:00.408486 21910 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.397248 21882 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.408815 21527 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:00.318717 21909 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:00.490639 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.452795 21735 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:00.318717 21662 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:00.504063 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 63258.2us
I20260709 12:18:00.476135 21864 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.319067 21417 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:00.513100 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92488.4us
I20260709 12:18:00.510749 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.486900 21577 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:00.453277 21859 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.517685 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:00.487612 21911 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.482509 21919 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:00.485548 21921 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:00.524857 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 102407us
I20260709 12:18:00.526995 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:00.482015 21910 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:00.539906 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 305384us
I20260709 12:18:00.518977 21577 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:00.544991 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:00.546545 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69632.6us
I20260709 12:18:00.547377 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:00.551090 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:00.553613 18370 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:18:00.519392 21527 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:00.521397 21864 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:00.544251 21930 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.553941 18370 pending_rounds.cc:85] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:00.558817 18370 pending_rounds.cc:107] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:18:00.559137 18370 raft_consensus.cc:2889] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:00.554668 21564 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:00.552124 21662 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.549734 21928 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:00.553284 21882 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.530539 21894 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:00.555119 21923 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:00.555177 21885 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.547022 21919 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.558079 21735 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.555150 21642 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:00.901028 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.600968 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.623351 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:00.639902 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92908us
W20260709 12:18:00.696065 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.662s (SENT)
I20260709 12:18:00.763345 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:00.913689 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:00.676232 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2167) took 1970 ms. Trace:
I20260709 12:18:00.913839 18441 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:00.914361 21885 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.915874 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:00.917166 21885 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:00.603578 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52533.9us
W20260709 12:18:00.767139 21922 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
W20260709 12:18:00.750360 18412 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.649s (SENT)
I20260709 12:18:00.724736 21864 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:00.919534 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 169232us
W20260709 12:18:00.836285 21900 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.628s	user 0.001s	sys 0.009s
I20260709 12:18:00.920305 21900 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.827556 21937 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.921500 18449 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:00.771060 18368 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:18:00.667412 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 110193us
I20260709 12:18:00.915009 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:00.915006 21938 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:00.921882 21937 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.913843 18443 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.875838 18446 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
W20260709 12:18:00.914973 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 218924us
W20260709 12:18:00.921505 18412 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.779s (SENT)
I20260709 12:18:00.927482 18441 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:00.914799 18297 rpcz_store.cc:276] 0709 12:17:58.705395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.748939 (+ 43544us) service_pool.cc:224] Handling call
0709 12:17:58.795876 (+ 46937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.823615 (+ 27739us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:58.960704 (+137089us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.198604 (+237900us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b18003a7fe0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:00.488491 (+1289887us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.495051 (+  6560us) log.cc:844] Serialized 26 byte log entry
0709 12:18:00.495300 (+   249us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:00.495351 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.495438 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.658614 (+163176us) raft_consensus.cc:1685] finished
0709 12:18:00.669988 (+ 11374us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.676148 (+  6160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59916,"prepare.queue_time_us":218684,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":73973,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":38676,"spinlock_wait_cycles":10264448,"thread_start_us":1519218,"threads_started":3,"wal-append.queue_time_us":1082805}
I20260709 12:18:00.927300 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.913271 18444 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:18:00.603972 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48467.2us
I20260709 12:18:00.930724 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:00.900862 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 345651us
I20260709 12:18:00.832113 21859 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.931957 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:00.929426 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.933045 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.713s (SENT)
W20260709 12:18:00.772360 21939 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:18:00.765848 21935 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:18:00.939023 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:00.943745 18443 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:00.657788 21857 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.944818 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.927992 21642 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.946542 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:00.948644 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.911550 21860 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.700908 21667 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.764847 21527 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:00.723697 21929 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:00.608883 21926 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:00.948530 18444 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:00.966522 21926 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:00.967134 21926 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:00.969017 21937 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.970139 21644 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:00.967094 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:00.933670 21882 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:00.971827 18444 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.974119 21882 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:00.951799 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.974928 18446 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:00.975540 18441 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:00.951936 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:00.949931 21928 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.954874 21910 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:00.956199 19006 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.730s (SENT)
I20260709 12:18:00.922379 21900 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.938416 21564 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:00.979808 21564 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:00.980266 21564 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:00.956647 21818 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:00.566224 21930 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.985268 21928 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:00.961886 21635 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.201s	user 0.012s	sys 0.000s
I20260709 12:18:00.963193 21919 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:00.947408 21859 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:00.965817 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.986282 21635 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.988415 21635 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:00.977921 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.963285 21729 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.216s	user 0.007s	sys 0.004s
I20260709 12:18:00.991503 21729 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:00.950822 21857 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.992342 21857 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.992475 21859 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:00.993945 21859 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:00.978865 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.991581 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.000648 18440 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:00.985342 21930 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:00.999804 18298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 601) took 1955 ms (client timeout 1662 ms). Trace:
I20260709 12:18:00.945453 21923 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:01.007637 18298 rpcz_store.cc:269] 0709 12:17:59.044609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.130454 (+ 85845us) service_pool.cc:224] Handling call
0709 12:18:00.999584 (+1869130us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540004cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:00.999712 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600895872}
W20260709 12:18:01.001034 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.959132 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44234.7us
I20260709 12:18:01.009370 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:01.010030 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.010196 21919 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.010644 21919 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:01.010784 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:00.960309 21860 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.011097 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.011776 18443 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:18:01.012646 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.012981 21860 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:01.013118 18443 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.958937 21848 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:00.761538 21577 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.014633 21577 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.015710 19043 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:01.017199 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 850) took 1738 ms. Trace:
I20260709 12:18:01.017381 21729 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:01.017594 19043 rpcz_store.cc:276] 0709 12:17:59.278554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.278719 (+   165us) service_pool.cc:224] Handling call
0709 12:17:59.279141 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.016082 (+1736941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.016977 (+   895us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400454a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:01.017143 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3993088768}
I20260709 12:18:00.960492 21929 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.977470 21667 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.004824 21398 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:01.006965 21916 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.795s	user 0.008s	sys 0.003s
I20260709 12:18:01.009544 21923 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.011044 21919 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.013707 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 851) took 1734 ms. Trace:
I20260709 12:18:01.053583 19036 rpcz_store.cc:276] 0709 12:17:59.279693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.279873 (+   180us) service_pool.cc:224] Handling call
0709 12:17:59.282980 (+  3107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.653534 (+370554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.013399 (+1359865us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54006202d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:01.013634 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773403904}
I20260709 12:18:01.002767 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.055478 18440 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:01.055716 21564 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.057057 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:01.002748 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 81280.1us
I20260709 12:18:01.058799 21564 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.060413 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.061213 18440 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:01.020005 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:00.059448 18369 rpcz_store.cc:276] 0709 12:17:58.652592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.832468 (+179876us) service_pool.cc:224] Handling call
0709 12:17:58.858918 (+ 26450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.891036 (+ 32118us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:17:58.924065 (+ 33029us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.746314 (+822249us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b18004be200. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:17:59.971502 (+225188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.971576 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:17:59.971646 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.971735 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.971904 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32823,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":18738,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":41801,"spinlock_wait_cycles":21319552,"thread_start_us":218558,"threads_started":1}
W20260709 12:18:01.032343 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87589.7us
I20260709 12:18:01.036250 21872 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:01.020781 21929 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.032895 21398 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.044706 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.069571 19035 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:01.069571 18444 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:01.047580 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.057269 19043 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:01.061771 18441 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:01.070478 19039 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:01.062085 21923 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:01.027292 18443 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
W20260709 12:18:01.064249 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53514.1us
I20260709 12:18:01.071643 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:01.071602 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:01.075497 19036 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:18:01.076921 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.077486 18441 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.078254 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:01.079643 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:01.082981 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 66478.7us
I20260709 12:18:01.064747 21727 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.067402 21568 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:01.083710 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:01.080643 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.079180 21722 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.091125 18440 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.093194 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:01.094713 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.095112 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.099766 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.100692 19035 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:01.100777 18298 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:01.080871 21894 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.103953 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.083817 21577 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:01.105680 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:01.126391 19038 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:01.126513 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45999.4us
I20260709 12:18:01.127650 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:01.090551 21727 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.107139 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.129225 18441 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:01.100052 21798 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:01.108480 21635 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:01.108882 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.109071 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.109357 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:01.109545 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.132665 18440 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:01.110787 21945 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:01.135673 21635 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.136060 21635 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:01.136389 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.112623 21943 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:01.113629 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:01.138899 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.139935 19041 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:01.089891 21859 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:01.140499 21859 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.141059 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.141820 19041 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:01.113834 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.142570 21859 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:01.114645 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:01.076926 21923 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.143087 19042 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:01.143388 21923 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.144420 21923 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.145738 21923 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.122442 18299 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.080621 21948 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:01.125834 21894 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.151726 21894 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.152921 21577 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.153299 21577 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:01.153641 21577 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.125588 21860 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:01.123703 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.158586 18298 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:01.120046 18444 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:18:01.124922 19006 leader_election.cc:336] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.961s (SENT)
I20260709 12:18:01.106463 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.132692 19036 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.132050 21798 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.160115 21798 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:01.160197 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.160887 18295 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:01.163441 21398 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.135748 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.164723 21398 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.164918 18291 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:01.165764 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.166405 18444 raft_consensus.cc:2376] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9. Re-sending same reply.
I20260709 12:18:01.134691 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.167325 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.137038 21635 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.168032 18444 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:01.167927 21857 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.168820 21635 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.169057 19040 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.170094 21857 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.136092 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.159782 19039 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:01.172286 21727 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.173059 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:01.175346 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.176003 18444 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:01.180912 19036 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:01.122970 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.181864 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.155436 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.183779 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:01.184451 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 62203.4us
I20260709 12:18:01.184998 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.185207 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.186131 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.186722 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.171167 19038 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:01.194730 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:01.194784 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.197062 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.197815 18440 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:01.202095 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.202795 18440 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.199971 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.205543 18294 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:01.132048 21916 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.206852 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:01.208773 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.208925 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:01.210608 18441 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:01.214176 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:01.197726 21841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.034s	sys 0.061s
I20260709 12:18:01.193562 21882 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:01.193714 21642 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:01.196864 21900 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.209336 21857 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:01.245103 21857 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.246009 21857 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.249032 18299 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:01.217221 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.251168 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.203687 21859 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:01.221508 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.198374 21943 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:01.217978 18412 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67a1783f49244bd5ba98721205d1ff6e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:01.226161 18294 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:01.218762 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 91222.4us
I20260709 12:18:01.224876 21900 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.227344 21946 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.219390 21347 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.209709 21916 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.227864 21952 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:01.229413 21928 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.230968 21568 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:01.234301 21915 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.916s	user 0.003s	sys 0.000s
I20260709 12:18:01.237670 21919 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.238987 21926 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.201408 21727 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:01.218799 21722 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:01.218480 21841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.034s	sys 0.061s
I20260709 12:18:01.271761 21916 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.211273 21945 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:01.257766 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69366.5us
I20260709 12:18:01.259317 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1850) took 3127 ms. Trace:
I20260709 12:18:01.288244 18442 rpcz_store.cc:276] 0709 12:17:58.131871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.227583 (+ 95712us) service_pool.cc:224] Handling call
0709 12:17:58.227920 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.613667 (+1385747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.259024 (+1645357us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003b10d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:01.259196 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3185301248}
I20260709 12:18:01.261693 21798 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.290939 21798 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:01.259018 21631 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:01.265694 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.294109 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.268810 21347 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:01.256119 21894 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.266059 21900 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.277702 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:01.259859 21943 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.277535 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.279803 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1857) took 1595 ms. Trace:
I20260709 12:18:01.303071 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.211426 21930 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:01.267824 21859 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:01.225703 21882 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.274766 21928 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:01.326713 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51058.9us
W20260709 12:18:01.275067 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80387.4us
I20260709 12:18:01.298169 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 693) took 1077 ms. Trace:
I20260709 12:18:01.329345 18370 rpcz_store.cc:276] 0709 12:18:00.221047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.221165 (+   118us) service_pool.cc:224] Handling call
0709 12:18:00.221493 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.622242 (+400749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.297924 (+675682us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54000c85d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:01.298109 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749211520}
I20260709 12:18:01.301932 21347 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.303596 18294 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:18:01.280625 21722 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.303278 18448 rpcz_store.cc:276] 0709 12:17:59.684174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.684281 (+   107us) service_pool.cc:224] Handling call
0709 12:18:01.279634 (+1595353us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:01.279728 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108802816}
I20260709 12:18:01.302764 18440 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:01.286629 21568 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.310598 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:01.288045 21945 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.312475 21909 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:01.298368 21926 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.331981 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.309365 21818 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:01.309068 21943 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:01.309084 18338 connection.cc:256] RPC call timeout handler was delayed by -1.6645678349998434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 12:18:01.313357 21631 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.341959 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1735) took 2313 ms. Trace:
I20260709 12:18:01.359800 19040 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:01.283946 21952 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.291507 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:01.380123 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 88891.5us
W20260709 12:18:01.369710 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.371533 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 97421.7us
I20260709 12:18:01.331604 21644 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.332501 21928 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.373142 18448 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:01.374218 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:01.375037 18338 connection.cc:256] RPC call timeout handler was delayed by -1.538567834999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 12:18:01.395352 18338 connection.cc:256] RPC call timeout handler was delayed by -1.43686783499993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:18:01.395699 18338 connection.cc:256] RPC call timeout handler was delayed by -1.4053678349998791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 12:18:01.375770 18374 rpcz_store.cc:276] 0709 12:17:59.028450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.622847 (+594397us) service_pool.cc:224] Handling call
0709 12:17:59.623171 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.662511 (+ 39340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.662562 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.672686 (+ 10124us) log.cc:844] Serialized 26 byte log entry
0709 12:17:59.928845 (+256159us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:59.928902 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.928990 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.039993 (+1111003us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b48001f14d8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6dda5aa1 00007f3c6dd9cb37 00007f3c6dd9c9ca 00007f3c6de9426f 00007f3c6de93f37 00007f3c6dea855c 00007f3c6dea8508 00007f3c6dea8488 00007f3c6dea8443 00007f3c6dea76ec 00007f3c6cdd1a96
0709 12:18:01.233638 (+193645us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:01.341767 (+108129us) raft_consensus.cc:1685] finished
0709 12:18:01.341786 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.341905 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10246,"raft.queue_time_us":39004,"raft.run_cpu_time_us":14134,"raft.run_wall_time_us":1373568,"spinlock_wait_cycles":126457088,"thread_start_us":255721,"threads_started":1,"wal-append.queue_time_us":553943}
W20260709 12:18:01.396116 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3531678349997946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 12:18:01.396473 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1866678349999802s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 12:18:01.335889 21930 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.359465 21722 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.366015 21945 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.389629 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:01.393245 21577 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:01.418177 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.418006 21631 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.312300 21929 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.419646 21577 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.420296 21560 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:01.420693 21560 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:01.421108 21560 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.421572 21930 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.421870 21946 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:01.397524 18300 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:18:01.414955 21925 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.424504 21925 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:01.425274 21925 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.426404 21925 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.427157 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 756) took 1150 ms. Trace:
I20260709 12:18:01.427610 18449 rpcz_store.cc:276] 0709 12:18:00.277240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.420178 (+142938us) service_pool.cc:224] Handling call
0709 12:18:01.426968 (+1006790us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54000d9b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:01.427103 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149896960}
W20260709 12:18:01.400177 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.401932 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:01.429564 18198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 211) took 22099 ms (client timeout 15000 ms). Trace:
I20260709 12:18:01.430691 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.431866 21577 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.432509 21841 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:01.429939 18198 rpcz_store.cc:269] 0709 12:17:39.330621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.335372 (+  4751us) service_pool.cc:224] Handling call
0709 12:18:01.429473 (+22094101us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:59.027835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:59.028583 (+   748us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:59.028640 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:17:59.028672 (+    32us) tablet.cc:662] Decoding operations
0709 12:17:59.029561 (+   889us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:59.029638 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:59.029667 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:17:59.030430 (+   763us) tablet.cc:701] Row locks acquired
0709 12:17:59.030482 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:17:59.030655 (+   173us) write_op.cc:270] Start()
0709 12:17:59.030796 (+   141us) write_op.cc:276] Timestamp: P: 1783599479030593 usec, L: 0
0709 12:17:59.030819 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:17:59.135428 (+104609us) log.cc:844] Serialized 2835 byte log entry
0709 12:18:00.031745 (+896317us) op_driver.cc:464] REPLICATION: finished
0709 12:18:00.446632 (+414887us) write_op.cc:301] APPLY: starting
0709 12:18:00.467952 (+ 21320us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:00.519317 (+ 51365us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:00.547636 (+ 28319us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:00.675491 (+127855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:00.675875 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:18:00.692272 (+ 16397us) write_op.cc:312] APPLY: finished
0709 12:18:00.779541 (+ 87269us) log.cc:844] Serialized 59 byte log entry
0709 12:18:00.779840 (+   299us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:00.779998 (+   158us) write_op.cc:454] Released schema lock
0709 12:18:00.780074 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:00.780608 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:00.780648 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":170592,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":376466,"apply.run_cpu_time_us":5195,"apply.run_wall_time_us":360875,"num_ops":7,"prepare.queue_time_us":317960,"prepare.run_cpu_time_us":32372,"prepare.run_wall_time_us":542731,"raft.queue_time_us":359226,"raft.run_cpu_time_us":3973,"raft.run_wall_time_us":43883,"replication_time_us":1000747,"thread_start_us":913593,"threads_started":4,"wal-append.queue_time_us":496750}]]}
I20260709 12:18:01.433126 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.433184 21841 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:01.376662 21909 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.410620 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.410638 21958 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:01.410929 18374 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:18:01.411280 21884 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:01.412001 21934 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:01.388034 21818 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.390307 21644 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.390781 21642 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:01.413647 21417 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.441644 19040 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:01.376014 21957 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:01.444726 18374 raft_consensus.cc:606] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:01.445734 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.458796 18374 raft_consensus.cc:614] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:01.449709 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44468.3us
W20260709 12:18:01.454473 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43956us
I20260709 12:18:01.456260 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:01.466550 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.467408 18299 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:01.467870 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.468479 18294 raft_consensus.cc:2376] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8. Re-sending same reply.
I20260709 12:18:01.447556 21884 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.446377 21926 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:01.480103 21577 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:01.457192 21841 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:01.446990 21948 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:01.444288 21958 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:01.462426 21957 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.457587 21644 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.449347 21934 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:01.516080 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.490643 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.494334 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.548311 18291 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:01.480932 21894 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.498705 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:01.518025 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 61864.7us
I20260709 12:18:01.518675 19040 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:01.487632 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.447178 21860 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:18:01.463685 21857 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:01.537585 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.538472 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.447255 21398 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:18:01.543044 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.566959 18292 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:01.487694 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.549655 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.494624 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.550834 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.569200 18291 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:01.486575 21884 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.554960 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:01.571916 18449 raft_consensus.cc:2376] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7. Re-sending same reply.
I20260709 12:18:01.559296 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.574777 18298 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:01.455750 21859 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.539108 21958 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.541205 21841 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:01.538347 21919 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.539024 21957 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.521134 21848 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:01.592023 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.592293 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.592571 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.567734 21577 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:01.607571 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 142671us
I20260709 12:18:01.608176 18298 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:18:01.608685 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:01.624518 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.625135 18298 raft_consensus.cc:2376] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7. Re-sending same reply.
I20260709 12:18:01.626154 21577 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.627863 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:01.571187 21916 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.594573 21882 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:01.570477 21948 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.633788 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.634167 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.680126 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.680783 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.682514 18373 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:01.679855 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.679481 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.676498 18370 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.680469 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.611598 21642 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:01.711042 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.584975 21818 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.716017 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:01.723301 18295 raft_consensus.cc:2368] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8.
I20260709 12:18:01.723613 18371 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:01.675411 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:01.726063 21964 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:18:01.726897 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.727856 18373 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:01.634859 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.674471 21727 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:01.747816 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.748337 21727 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.748422 21642 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.749060 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.749327 21727 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.727043 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:01.750936 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.706245 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.731405 21872 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:01.752411 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.753983 21872 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:01.735708 21959 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:18:01.738113 21968 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 12:18:01.739215 21969 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:18:01.739703 21856 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.757128 21856 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.757973 21856 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.741567 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.759416 18295 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:01.736074 21560 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.745163 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.745930 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.764130 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.746677 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.624053 21568 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:01.699077 21667 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:01.749884 21971 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.165s	user 0.003s	sys 0.000s
I20260709 12:18:01.754087 18370 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:01.742744 21929 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:01.761623 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 199077us
W20260709 12:18:01.731330 21962 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:18:01.671792 21635 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:01.763267 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 128484us
I20260709 12:18:01.764390 18293 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.611522 21972 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.157s
I20260709 12:18:01.764775 18300 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:01.767235 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.769300 18295 raft_consensus.cc:2376] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
I20260709 12:18:01.767709 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.759096 21856 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.770133 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.770855 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.711962 21931 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.771847 18298 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:01.759701 21923 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.002s
I20260709 12:18:01.576129 21909 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.775094 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.785356 18295 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:01.786281 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.776146 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.799254 18294 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:01.799410 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.729048 21952 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.766217 21925 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:01.788098 18296 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:01.807299 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.754827 21872 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.813035 18292 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:01.807580 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.619669 21919 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.156s
I20260709 12:18:01.801558 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.810062 18299 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:01.767174 21667 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:01.811911 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.774609 21931 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.850504 18295 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:01.823136 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.851884 18294 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:01.851812 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.623476 21934 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.822115 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.857146 18296 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.819196 18300 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:01.771986 21930 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:18:01.859112 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:01.772964 21635 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:01.849255 21900 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
W20260709 12:18:01.846719 21962 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:01.806257 21926 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.868369 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.815470 21925 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:01.869793 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 98827us
I20260709 12:18:01.872648 18293 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:01.879415 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.871034 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:01.874859 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 300642us
W20260709 12:18:01.864702 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 134637us
I20260709 12:18:01.876304 18373 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:01.877132 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:01.861661 21857 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:01.774250 21841 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:01.886503 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:01.887074 18372 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.889525 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.890275 18293 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:01.891142 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.814574 21347 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.894299 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.895193 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.895929 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:01.899508 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.900187 18373 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:01.901081 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.901644 18373 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
W20260709 12:18:01.882907 21973 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:18:01.898999 18370 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:01.884629 21925 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:01.907938 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.907157 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.889416 21946 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:01.890028 21952 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.920892 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.920981 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.922812 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.922887 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.923578 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.876107 21900 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.927211 18368 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:01.929915 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.928220 18373 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:01.936226 18370 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:01.887974 21860 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:18:01.881325 21722 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.956555 21988 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.957059 21988 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.957386 21988 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.959622 21560 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:01.959357 21857 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:01.960013 21978 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:18:01.960661 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.898075 21644 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:01.964905 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:01.966774 19006 leader_election.cc:336] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.754s (SENT)
W20260709 12:18:01.970907 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 80932.9us
I20260709 12:18:01.974148 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.974797 18373 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:01.975986 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.977150 18295 raft_consensus.cc:2410] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:18:01.979828 19006 leader_election.cc:336] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.088s (SENT)
I20260709 12:18:01.982211 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:01.987629 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.008338 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.008462 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.008931 18373 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
W20260709 12:18:01.991672 18415 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.587s (SENT)
I20260709 12:18:02.010291 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:01.924695 21975 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:01.992990 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.028719 19040 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.987197 18372 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.030544 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.032027 18371 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.926797 21952 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.007045 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 691) took 1786 ms. Trace:
W20260709 12:18:02.001464 19006 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.793s (SENT)
I20260709 12:18:01.933180 21946 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:01.937584 21916 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.009317 21948 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:02.084335 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92723.2us
I20260709 12:18:02.089382 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:02.090703 18415 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.610s (SENT)
I20260709 12:18:02.090930 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:01.937412 21931 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:01.988494 21979 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:18:02.102567 18297 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:18:02.096836 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:02.116892 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 115380us
I20260709 12:18:02.091827 18296 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:02.117509 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.090355 18367 rpcz_store.cc:276] 0709 12:18:00.220264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.248037 (+ 27773us) service_pool.cc:224] Handling call
0709 12:18:00.283756 (+ 35719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.502380 (+218624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.502409 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.507623 (+  5214us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.459755 (+952132us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b18004be3e0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:01.551183 (+ 91428us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:01.578735 (+ 27552us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.766631 (+187896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.766756 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:02.006773 (+240017us) raft_consensus.cc:1685] finished
0709 12:18:02.006870 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.006992 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7285,"prepare.queue_time_us":28187,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4994,"raft.queue_time_us":331700,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":21044,"spinlock_wait_cycles":2828544,"thread_start_us":452845,"threads_started":3,"wal-append.queue_time_us":1104193}
W20260709 12:18:02.092572 21945 consensus_meta.cc:287] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.001s
I20260709 12:18:01.988624 21642 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.998777 21980 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:02.142136 18297 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:02.091392 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:02.143041 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.366s (SENT)
I20260709 12:18:02.142940 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.143674 18299 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.165385 19040 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:02.156126 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.089422 21952 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.168924 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:02.141503 21928 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.000s
I20260709 12:18:02.093113 21722 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:02.165570 21921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.012s	sys 0.032s
W20260709 12:18:02.093122 21943 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.000s
I20260709 12:18:02.043903 21974 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:02.173267 19006 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.878s (SENT) [suppressed 57 similar messages]
I20260709 12:18:02.137101 21984 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:02.169922 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.078786 21958 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.136055 21938 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.063846 21925 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.180266 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.368052 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:02.183320 21983 ts_tablet_manager.cc:933] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.183209 21943 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.045439 21957 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.192082 21900 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.196897 21635 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:02.166983 21980 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:02.096304 21946 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.136035 21560 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:18:02.356276 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.396315 18293 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.397262 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.398036 18293 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.182765 21928 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.400061 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.400305 18368 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.400677 21934 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:02.399672 18375 raft_consensus.cc:2368] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:02.340890 21894 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:02.412626 21894 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:02.413388 21928 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.325856 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.413625 21894 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.414466 18295 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.169423 21642 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.413636 21667 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.415620 21894 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:02.415846 21667 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:02.323194 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.416347 21965 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.903s	user 0.002s	sys 0.002s
W20260709 12:18:02.417225 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.364568 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.365787 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:02.166213 21945 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.238478 18374 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.093922 21975 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:02.322361 18298 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.368196 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.368650 18373 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:18:02.369115 18441 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:02.204947 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 114263us
I20260709 12:18:02.364568 19040 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:02.181458 21921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.032s
I20260709 12:18:02.280727 21958 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:02.349021 21417 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.885s	user 0.008s	sys 0.003s
I20260709 12:18:02.375701 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:02.377784 18442 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:02.378858 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:02.379300 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:02.246398 21884 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.661s	user 0.001s	sys 0.006s
I20260709 12:18:02.193892 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:02.281965 21998 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:18:02.478289 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.477221 18440 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:02.262310 21993 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:18:02.386458 21957 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.389369 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.483466 18296 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:02.484469 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.352628 21988 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.303413 21916 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.234927 21984 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.177497 21977 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.392115 21980 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.393262 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.403069 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.236590 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.544168 18368 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.544740 18299 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:02.544896 18292 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.407526 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.407637 18449 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:02.409521 18367 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.409585 21943 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.411311 21925 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.373858 21919 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.412034 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.414310 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.305616 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:02.418056 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.549592 19043 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.418289 18444 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.421271 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.432245 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.433604 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.441826 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:02.428984 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51919.5us
I20260709 12:18:02.453433 18369 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.477219 18366 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.476702 18447 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
W20260709 12:18:02.478219 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110103us
I20260709 12:18:02.479943 18373 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.387847 18446 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:02.416388 21642 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.478259 18443 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:02.478473 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:02.482129 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.485893 18291 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:02.502494 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.506362 21644 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.513275 21347 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.515679 21560 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:02.515823 21568 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:02.518601 21931 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.563331 21560 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:02.556766 21970 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:18:02.480139 21648 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.406706 18448 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:02.546293 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.549566 18293 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.556564 18296 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:02.481122 21884 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.559073 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:02.561105 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 128953us
I20260709 12:18:02.557361 18374 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 5.
W20260709 12:18:02.589542 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 156034us
I20260709 12:18:02.589572 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.590673 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.591001 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.592142 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.569506 18440 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:18:02.572384 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:02.573001 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:02.573622 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 131906us
I20260709 12:18:02.593464 18444 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:02.594394 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.520875 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1868) took 1974 ms. Trace:
I20260709 12:18:02.577445 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.581786 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:02.558413 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:02.596457 18375 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.560700 21947 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.598212 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.615096 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:02.616068 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.616514 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.595419 18445 rpcz_store.cc:276] 0709 12:18:00.546150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.171932 (+625782us) service_pool.cc:224] Handling call
0709 12:18:01.436569 (+264637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.445866 (+  9297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.445914 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.461515 (+ 15601us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.880680 (+419165us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:01.880709 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.880753 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:02.095185 (+214432us) raft_consensus.cc:1685] finished
0709 12:18:02.095204 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.520552 (+425348us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54002aa150. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:02.520798 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8878,"raft.queue_time_us":10624,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":8278,"spinlock_wait_cycles":607814400,"thread_start_us":351785,"threads_started":1,"wal-append.queue_time_us":458479}
I20260709 12:18:02.570405 21980 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:02.621369 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63063.2us
I20260709 12:18:02.622253 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:02.622879 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63857.3us
I20260709 12:18:02.611694 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:02.538597 21916 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.570287 21568 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:02.564536 21848 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.628618 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.632649 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.637907 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.583349 21648 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.633718 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.634477 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.579473 21974 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:02.621229 21928 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.579571 21938 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:18:02.636125 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.620922 21798 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.646427 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.638715 18374 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:02.636678 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:02.638810 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:02.639086 19006 leader_election.cc:400] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8.
I20260709 12:18:02.583841 21642 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.633435 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.564218 21919 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.641623 18448 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:02.636586 21958 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.644212 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.647531 19006 leader_election.cc:403] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:02.648186 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50043.2us
I20260709 12:18:02.637049 21977 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.579991 21983 tablet_copy_client.cc:323] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:18:02.648942 18369 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:02.650213 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:02.675730 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85111.5us
I20260709 12:18:02.676508 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.677922 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:02.615425 21925 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.635900 21856 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:02.679409 21856 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:02.679888 21856 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.679955 21884 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.680476 21925 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:02.680934 21925 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.680830 21568 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.681881 21568 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.636168 21921 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:02.653160 22006 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 12:18:02.683467 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60774.2us
I20260709 12:18:02.643478 21648 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.684579 21988 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.684971 21919 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.685380 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.686820 21977 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:02.655370 21944 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:02.687288 21977 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:02.687613 21860 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.687750 21977 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:02.688001 21347 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:02.688416 21860 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.688589 21347 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:02.689555 21347 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.585197 21975 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.649534 21642 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:02.690819 21642 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.687691 21944 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:02.651259 21577 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.693359 21577 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:02.693763 21577 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.693465 21944 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.695454 21921 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:02.696513 21916 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.696409 21921 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.674937 21945 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:02.648120 21946 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.636157 21560 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.700062 21560 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:02.701015 21923 consensus_queue.cc:1175] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" responder_term: 10 status { last_received { term: 10 index: 10 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:02.640553 21417 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.637306 21735 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:02.666939 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.583061 21952 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.669776 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:02.718441 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48750.3us
I20260709 12:18:02.625556 21943 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.717896 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:02.670665 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:02.722445 18375 raft_consensus.cc:2368] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
I20260709 12:18:02.623420 21635 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:02.673290 18368 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.674715 21957 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.687011 21930 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8.
I20260709 12:18:02.699427 21984 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:02.628018 21926 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:18:02.673635 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.708879 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.716816 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:02.717593 18373 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:02.732128 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58546.3us
I20260709 12:18:02.734118 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:02.635360 21947 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.707800 21946 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.745231 18370 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.743726 18368 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:02.740945 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:02.717871 21916 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.754817 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.751046 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.756640 19039 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:02.755095 18373 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:02.760834 19036 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:02.761396 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.729822 21925 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.742362 21945 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:02.767871 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:02.769685 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.769542 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:02.750033 21798 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.749915 21860 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.759141 21417 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.779937 19039 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:02.781481 18375 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:02.782701 19036 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:02.783314 21635 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.792622 18375 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.745766 21872 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:18:02.796813 19037 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:02.771071 21958 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:02.797839 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47238.2us
I20260709 12:18:02.774459 21945 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:02.785379 21417 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:02.806634 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:02.763506 21916 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.790956 21635 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.826390 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.823038 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.828254 18299 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.834127 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:02.869510 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.876287 19037 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:02.876853 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.868593 22014 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:02.877964 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.878266 22014 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:02.878486 18297 raft_consensus.cc:2376] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10. Re-sending same reply.
I20260709 12:18:02.878808 22014 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.879448 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.880003 18297 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:02.871253 22011 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:18:02.883828 22015 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:02.885718 21916 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.886401 22014 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.887252 18297 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:02.887084 21648 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.888140 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.888432 22014 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.888705 18297 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:02.889430 21648 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:02.889436 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:02.890105 18300 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:02.890996 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.891899 18299 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:02.872044 21952 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.873276 21577 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.874442 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.824517 21945 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.800246 21974 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:02.823872 21417 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.831650 18296 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:02.797427 21900 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:02.878311 18368 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:02.893088 19043 raft_consensus.cc:2368] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:18:02.895100 18292 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260709 12:18:02.896207 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 174418us
I20260709 12:18:02.897169 18299 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:02.827651 21894 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.905391 18300 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:02.906239 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.907858 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.908314 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.908859 18299 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 6: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:02.830485 21635 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.824388 22012 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:02.904073 18292 pending_rounds.cc:85] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 7
I20260709 12:18:02.910321 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.943960 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.912926 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:02.946950 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.945111 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.947108 18297 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:02.944684 18292 pending_rounds.cc:107] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:18:02.948618 18292 raft_consensus.cc:2889] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:02.948554 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:02.898504 21916 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.949343 18299 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.913297 18374 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:02.950858 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 242063us
I20260709 12:18:02.952694 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.955540 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.894054 21977 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:02.912045 21934 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.004s
W20260709 12:18:02.957643 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.960556 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:02.949681 22012 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.910068 21894 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:02.966796 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.968324 21930 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:02.968684 22026 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.951001 21722 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.955416 21938 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:02.979765 21722 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.979960 21938 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:02.980304 21722 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.980901 21938 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.981508 21894 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.953945 21735 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:18:02.983017 18294 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:18:02.971740 22020 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:02.984923 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1757) took 1021 ms. Trace:
I20260709 12:18:02.985636 21894 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:02.986495 21974 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.985303 18372 rpcz_store.cc:276] 0709 12:18:01.963436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.096579 (+133143us) service_pool.cc:224] Handling call
0709 12:18:02.984720 (+888141us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54002df0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:02.984865 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484291328}
W20260709 12:18:02.989346 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.913718 21648 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.971999 21564 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:02.956545 21916 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:02.969630 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:02.997879 18299 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
W20260709 12:18:03.013201 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43637.5us
I20260709 12:18:03.013947 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:02.963797 21984 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:02.975404 21818 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:02.976559 22016 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.977603 22012 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:02.909471 21642 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:02.975373 22017 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:03.013206 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:03.017786 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 161158us
I20260709 12:18:03.032262 19037 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
W20260709 12:18:03.019748 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65834.2us
I20260709 12:18:02.992360 22023 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:03.026018 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 71517.4us
I20260709 12:18:03.040113 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:03.038954 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:03.041090 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.042704 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:03.042671 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 90156.6us
I20260709 12:18:03.019057 21942 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.044866 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.042330 21667 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.049475 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:03.030514 21930 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:03.053053 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:03.031968 21818 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:03.030334 21925 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:03.056759 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.058676 21667 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.059762 21930 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:03.058104 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.021093 21884 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:03.039698 22023 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:03.051339 21872 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:03.067780 21818 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.068809 21952 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:03.071290 18444 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:03.071928 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:03.061072 22018 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:18:03.090854 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.076197 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:03.095866 18441 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:03.062186 21560 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:03.098361 18448 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:18:03.099054 18448 pending_rounds.cc:85] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 4
I20260709 12:18:03.077792 18442 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:03.099371 18448 pending_rounds.cc:107] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:18:03.099778 18448 raft_consensus.cc:2889] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:03.101564 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 43559.5us
I20260709 12:18:03.066285 22016 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:03.102741 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:03.063259 21900 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.105014 18299 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:03.106179 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.104935 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:03.108202 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51538.6us
I20260709 12:18:03.109351 18415 leader_election.cc:400] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:03.061918 21974 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:03.109766 18415 leader_election.cc:403] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:03.080606 21872 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:03.119256 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.115222 21856 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.125444 18293 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.127626 21894 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.132555 21894 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.132913 21894 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.133044 21722 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.135496 21996 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:03.137215 22021 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:03.137909 21417 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.120915 21872 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:03.139887 22025 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:18:03.117432 21560 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:03.140945 21560 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.142470 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.143443 18299 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:03.139770 21900 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.149044 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1769) took 1410 ms. Trace:
I20260709 12:18:03.149845 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1758) took 1186 ms. Trace:
I20260709 12:18:03.151048 18291 rpcz_store.cc:276] 0709 12:18:01.739012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.746425 (+  7413us) service_pool.cc:224] Handling call
0709 12:18:03.148854 (+1402429us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b540004db54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:03.148989 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1657373440}
I20260709 12:18:03.151149 18367 rpcz_store.cc:276] 0709 12:18:01.963679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.141721 (+178042us) service_pool.cc:224] Handling call
0709 12:18:03.149677 (+1007956us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400394c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:03.149796 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544668032}
I20260709 12:18:03.152429 18369 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:03.152719 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.116968 21974 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:03.163249 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1779) took 1421 ms. Trace:
I20260709 12:18:03.164119 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1756) took 1201 ms. Trace:
I20260709 12:18:03.164489 18370 rpcz_store.cc:276] 0709 12:18:01.963164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.008172 (+ 45008us) service_pool.cc:224] Handling call
0709 12:18:03.163965 (+1155793us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:03.164061 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1692632960}
I20260709 12:18:03.163637 18298 rpcz_store.cc:276] 0709 12:18:01.741798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.799038 (+ 57240us) service_pool.cc:224] Handling call
0709 12:18:03.163070 (+1364032us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400330254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:03.163173 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121801088}
I20260709 12:18:03.170027 22029 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.171693 21916 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.173105 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 67a1783f49244bd5ba98721205d1ff6e from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:18:03.173705 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 67a1783f49244bd5ba98721205d1ff6e from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-67a1783f49244bd5ba98721205d1ff6e
I20260709 12:18:03.175541 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.129894 21577 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.164891 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:03.179106 18447 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:03.184803 18298 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:03.183712 18371 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:18:03.185510 18371 pending_rounds.cc:85] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:03.185711 18371 pending_rounds.cc:107] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:03.185849 18371 raft_consensus.cc:2889] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:03.144433 22028 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.156941 21722 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:03.193858 18292 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:03.187639 21977 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:03.163616 22017 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:03.224359 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:03.225174 19037 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:03.226184 21642 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:03.190443 22016 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:03.153347 21930 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.227789 22016 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.165855 21974 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:03.128110 21946 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:18:03.249339 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.253801 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:03.256132 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:03.256951 18372 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:03.258680 18368 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:03.258814 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.259949 18300 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.261093 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:03.261656 18372 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:03.267606 18368 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:03.278875 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:03.299171 18368 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.299448 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:03.300410 18372 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:03.231750 21931 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:03.301501 19038 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:03.302277 19040 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:03.309998 18370 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
W20260709 12:18:03.313575 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.316375 18440 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.268702 21938 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:03.303313 22032 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:18:03.303830 21931 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:03.337539 22030 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 12:18:03.338881 21980 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
W20260709 12:18:03.339116 22031 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:18:03.339545 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:03.339985 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:03.355264 18367 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:03.346781 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 241893us
I20260709 12:18:03.349888 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:03.350325 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:03.259326 22012 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.367097 18366 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:18:03.375120 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 113403us
I20260709 12:18:03.405333 18368 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:03.409245 21667 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.405375 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.422261 18300 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:03.350440 21931 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:03.184021 22023 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.432452 18449 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:03.437242 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:03.353331 22032 consensus_queue.cc:1243] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:03.441023 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.445592 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.411451 21980 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.355319 21952 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:18:03.453312 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:03.363809 21975 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:18:03.351349 21872 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:03.467111 18339 leader_election.cc:336] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.553s (SENT)
W20260709 12:18:03.479442 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.457757 22023 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.423362 21938 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:18:03.504768 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.505407 18339 leader_election.cc:336] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.594s (SENT)
I20260709 12:18:03.507433 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.425601 21631 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:18:03.506758 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 156502us
I20260709 12:18:03.457221 21980 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.513311 18443 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:03.513784 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.516443 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:03.517200 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.517333 18338 leader_election.cc:400] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:18:03.518179 18338 leader_election.cc:403] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:03.469043 22032 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.439599 21884 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 12:18:03.529271 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 420071us
I20260709 12:18:03.464078 21642 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:03.525070 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.520159 18373 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:03.529982 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.484848 21977 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.533165 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.489104 22029 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:03.537762 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.507454 21952 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:03.519726 21980 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:03.535598 22035 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:18:03.538791 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:03.537015 21928 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7.
I20260709 12:18:03.569613 21947 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:03.570969 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.600075 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.601797 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 717) took 1231 ms. Trace:
I20260709 12:18:03.539182 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.530073 21975 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.535741 22039 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:03.534627 21977 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.444746 21930 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:18:03.560873 21644 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:03.602387 18371 rpcz_store.cc:276] 0709 12:18:02.370410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.766389 (+395979us) service_pool.cc:224] Handling call
0709 12:18:02.783568 (+ 17179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.186113 (+402545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.601488 (+415375us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400115b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:03.601716 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907640064}
I20260709 12:18:03.537246 22032 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.546285 21568 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:18:03.564780 21642 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:03.628782 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 103562us
I20260709 12:18:03.611641 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:03.631824 18412 leader_election.cc:336] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.179s (SENT)
I20260709 12:18:03.632738 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.634238 21872 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:03.634793 21945 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.001s
W20260709 12:18:03.607834 22044 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.006s
I20260709 12:18:03.637230 19039 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:03.610680 21644 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:03.638441 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:03.638411 21644 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.638882 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:03.609751 21977 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.635161 21945 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.641916 22032 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:03.643277 21945 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.644690 22017 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:03.645543 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.646699 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.654325 18441 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:03.609586 22039 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:03.655623 22039 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.656400 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.657487 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:03.658550 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.639801 21644 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.662216 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:03.625465 22038 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.110s	user 0.006s	sys 0.000s
I20260709 12:18:03.629494 21642 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:03.625327 21635 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.674s	user 0.001s	sys 0.014s
W20260709 12:18:03.702404 21894 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.001s
I20260709 12:18:03.702112 21635 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.702816 21894 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.703557 22054 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.685017 21642 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.706405 21894 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.707530 21931 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.685240 22012 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:03.709535 18292 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:03.610039 22042 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.080s	user 0.004s	sys 0.003s
W20260709 12:18:03.663617 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63639.8us
I20260709 12:18:03.664809 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:03.720245 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55485.5us
I20260709 12:18:03.720460 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.663808 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.610421 21930 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:03.731429 21930 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:03.732486 21930 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.734047 21930 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.607504 21946 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:18:03.665572 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.735672 21946 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:03.736011 21946 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:03.737198 21946 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.738293 21946 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:03.673674 22017 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.607370 22050 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.739696 22050 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.742259 22050 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.612960 22043 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:03.688858 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56199.8us
I20260709 12:18:03.757886 18448 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.728050 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.685040 22057 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:03.728579 22012 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:03.821787 22012 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.729008 21931 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.821802 22057 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:03.823856 22057 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:03.824672 22057 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.825857 22057 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.730265 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.730913 21928 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.747769 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.679675 18459 tablet_copy_source_session.cc:215] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:18:03.819628 22053 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:03.820741 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92788.1us
I20260709 12:18:03.879192 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.880992 18444 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:03.888934 18445 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:03.892136 18415 leader_election.cc:400] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
I20260709 12:18:03.892611 18415 leader_election.cc:403] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:03.894644 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.897035 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 366063us
I20260709 12:18:03.827859 21996 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:03.916136 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 194935us
I20260709 12:18:03.757507 22028 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.920953 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.918797 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:03.892078 22017 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.922700 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.927915 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:03.928797 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 405042us
I20260709 12:18:03.931691 18293 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:03.934576 18297 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:03.913681 22026 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8.
I20260709 12:18:03.937296 18448 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:03.937481 18295 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:03.933141 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.922123 21996 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.942041 21996 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:03.933429 21952 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.943771 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.943954 21980 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.944581 18293 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:03.927156 21975 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:03.921903 22061 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:18:03.925041 21894 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:18:03.950838 18372 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:03.951771 21980 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:03.951917 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59912.2us
I20260709 12:18:03.953429 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.953733 18449 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.954427 22053 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:03.955164 22012 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.955593 22053 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:03.955963 22053 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:03.956686 21916 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:03.957157 21916 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:03.956796 22053 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.958155 22053 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.957973 21916 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.959532 21925 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:03.959908 21925 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:03.960680 21925 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:03.962009 21925 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:03.969312 22017 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.970285 22063 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:03.971544 22061 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:03.971939 22061 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:03.972761 22061 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.973899 21417 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:03.974148 22062 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:03.974167 22061 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.974923 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 898) took 1375 ms. Trace:
I20260709 12:18:03.975328 19038 rpcz_store.cc:276] 0709 12:18:02.599888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.600482 (+   594us) service_pool.cc:224] Handling call
0709 12:18:02.600838 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:03.302059 (+701221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.974693 (+672634us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540066e4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:03.974862 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610522496}
I20260709 12:18:03.976179 22064 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:03.976605 22021 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:03.977039 22021 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:03.978295 22021 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:03.978744 21635 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:03.979525 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56984.3us
I20260709 12:18:03.921674 21945 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.982703 18297 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:18:03.982769 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.983523 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.986400 18296 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:03.986588 18293 raft_consensus.cc:2376] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7. Re-sending same reply.
I20260709 12:18:03.987409 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.955011 18449 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:03.984045 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.989333 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:03.989595 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.954970 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:03.990327 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.990622 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:03.990852 18299 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:03.991200 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58112.9us
I20260709 12:18:03.991892 18368 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:03.991925 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:03.992587 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 625) took 1171 ms. Trace:
I20260709 12:18:03.992902 18294 rpcz_store.cc:276] 0709 12:18:02.821503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.821662 (+   159us) service_pool.cc:224] Handling call
0709 12:18:03.985735 (+1164073us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b54002760d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:03.992530 (+  6795us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188632320}
I20260709 12:18:03.959085 21916 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.999905 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:04.000170 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:03.930138 22052 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.004308 18300 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
W20260709 12:18:04.007557 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52682.8us
I20260709 12:18:04.007929 18293 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:04.008488 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.010315 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:04.005774 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.012756 18449 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:04.013728 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.013815 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:04.020603 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:03.989334 21417 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:04.024175 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:04.025651 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.027284 18295 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:04.005250 22021 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.036854 22057 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.040817 18441 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.001614 21952 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.006179 21635 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.006681 21980 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:04.059149 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90864.2us
I20260709 12:18:04.006423 22052 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.050913 18299 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:04.011596 22050 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.017158 21975 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:04.025104 22023 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 12:18:04.061374 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:04.070432 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:04.022049 22026 raft_consensus.cc:2764] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:03.996770 21945 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:04.059497 21974 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.000s
I20260709 12:18:04.086611 22064 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.063958 21347 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.067204 22052 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.072369 21952 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.043650 21921 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:18:04.049322 21417 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.059418 21977 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.104310 19037 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:04.087419 21980 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.108990 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.109843 18447 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:18:04.078622 21722 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:04.067035 22039 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.073503 22050 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.112753 18443 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:04.074270 21729 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:18:04.103259 21900 raft_consensus.cc:2764] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:04.117286 18446 raft_consensus.cc:2368] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8.
I20260709 12:18:04.082216 21974 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.129132 21977 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:04.131946 22050 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.133638 22039 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.142004 22043 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.142693 22043 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.144028 18371 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:04.100761 21347 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.145735 21417 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.147022 21347 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:04.148202 18370 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.150177 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:04.151317 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:04.152325 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.159420 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:04.156966 21974 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.191747 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.103348 21642 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.160154 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.097131 22064 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.203126 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.164721 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:04.204311 18298 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.143169 22043 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.174196 21635 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.175410 18441 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:04.125936 22017 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:18:04.182016 22052 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.104025 22057 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:18:04.187897 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.157424 18440 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:04.164081 21945 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.211005 18297 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:04.225286 18449 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:04.226480 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.238009 18298 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
W20260709 12:18:04.239075 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.240208 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 129611us
I20260709 12:18:04.241739 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:04.242899 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.243614 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.244557 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.242347 18442 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:04.246538 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.230746 21916 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.247099 18442 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:04.247570 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.233304 22050 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.252998 21945 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.236215 22017 consensus_queue.cc:1243] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:04.255571 21925 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.230376 21931 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:04.249039 21916 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.296131 22064 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.296376 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 725) took 1029 ms. Trace:
I20260709 12:18:04.283998 21916 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.296715 18372 rpcz_store.cc:276] 0709 12:18:03.266660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.411191 (+144531us) service_pool.cc:224] Handling call
0709 12:18:04.296192 (+885001us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54002e7a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.296321 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178848512}
W20260709 12:18:04.298163 18366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 710) took 2108 ms (client timeout 2087 ms). Trace:
W20260709 12:18:04.298583 18366 rpcz_store.cc:269] 0709 12:18:02.189666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.599166 (+409500us) service_pool.cc:224] Handling call
0709 12:18:04.297953 (+1698787us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b5400168ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.298088 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1656538368}
I20260709 12:18:04.302192 21944 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.302465 21642 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:04.302628 21944 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:04.287186 22078 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.303905 22078 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.305992 22078 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.312357 18445 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:04.314217 18448 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:18:04.312505 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:04.316700 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.320438 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:04.321203 18298 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:04.258244 22050 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:04.324990 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69626.9us
I20260709 12:18:04.325845 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:04.322158 18448 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:04.327481 18448 pending_rounds.cc:85] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:04.327708 18448 pending_rounds.cc:107] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:18:04.311319 22057 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:04.323307 21931 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:04.328029 18448 raft_consensus.cc:2889] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:04.332129 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.336267 18445 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:04.337188 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:04.339776 22077 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:18:04.333207 22057 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.346357 22057 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.333017 21996 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.347101 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 2) took 1903 ms. Trace:
I20260709 12:18:04.347170 21996 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.347872 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2216) took 1019 ms. Trace:
I20260709 12:18:04.348271 18292 rpcz_store.cc:276] 0709 12:18:03.328146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.328294 (+   148us) service_pool.cc:224] Handling call
0709 12:18:04.347659 (+1019365us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.347808 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735734528}
I20260709 12:18:04.331728 21931 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.349718 21931 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.349576 21996 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.350682 21856 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.352710 21946 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.353119 21946 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:04.353893 21946 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.333513 22078 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.356516 22078 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.356930 22078 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.357143 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 805) took 1185 ms. Trace:
I20260709 12:18:04.357525 18440 rpcz_store.cc:276] 0709 12:18:03.171876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.318663 (+146787us) service_pool.cc:224] Handling call
0709 12:18:03.318964 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.224488 (+905524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.356861 (+132373us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:04.357076 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927824768}
I20260709 12:18:04.358781 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.359997 18448 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:04.362751 22043 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.366092 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.366753 19037 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
W20260709 12:18:04.368216 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.369019 22037 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.958s	user 0.011s	sys 0.007s
I20260709 12:18:04.347697 19040 rpcz_store.cc:276] 0709 12:18:02.443297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.443668 (+   371us) service_pool.cc:224] Handling call
0709 12:18:04.346924 (+1903256us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540066e4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.347008 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714597632}
I20260709 12:18:04.371357 21958 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.371706 21958 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:04.372394 21958 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.375181 18449 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.376565 18440 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:04.378253 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:04.366261 18196 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:04.382764 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.384719 18448 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:04.384578 21930 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:04.385156 21930 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.386173 22050 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.328543 21944 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.387161 21930 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.387424 21944 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:04.390921 21984 consensus_queue.cc:1175] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3da7ad8fa20a4591963b660e4c848e7f" responder_term: 8 status { last_received { term: 7 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:04.391523 21974 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:04.392040 21974 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:04.393091 21974 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.393828 21568 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.394376 21568 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.394613 21974 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.395416 22043 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.395819 22043 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.396198 22043 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.396476 21926 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.396943 21926 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.401257 21568 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.403311 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.403703 19038 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:04.405130 18370 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:04.406473 22052 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.406533 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.407734 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 711) took 2217 ms. Trace:
I20260709 12:18:04.408766 18370 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:04.408236 18369 rpcz_store.cc:276] 0709 12:18:02.189909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.698358 (+508449us) service_pool.cc:224] Handling call
0709 12:18:03.152194 (+453836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.152963 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.407502 (+1254539us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b5400394c50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.407664 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043083648}
W20260709 12:18:04.409936 18373 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1769) took 1657 ms (client timeout 2137 ms). Trace:
W20260709 12:18:04.410475 18373 rpcz_store.cc:269] 0709 12:18:02.753018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.757400 (+  4382us) service_pool.cc:224] Handling call
0709 12:18:04.409734 (+1652334us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54000f8a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.409867 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714351360}
I20260709 12:18:04.411711 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.412520 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.412719 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1900) took 2040 ms. Trace:
I20260709 12:18:04.414177 21884 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:04.414593 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.373365 21958 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.415522 21644 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:04.411724 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.373129 21931 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.418126 21931 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.420240 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:04.420387 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:04.420830 18299 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:04.421931 21667 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:04.422350 21667 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:04.423590 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.398990 21926 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.420270 18292 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:18:04.413098 18449 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:04.429250 19038 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.430239 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.423099 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:04.413048 18375 rpcz_store.cc:276] 0709 12:18:02.371843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.793666 (+421823us) service_pool.cc:224] Handling call
0709 12:18:04.411797 (+1618131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.411973 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.411994 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:04.412028 (+    34us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:04.412060 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.412103 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.412574 (+   471us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54000f8a50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.412666 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3720561536}
I20260709 12:18:04.436738 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.437666 18375 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:04.439038 18297 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:04.420868 18293 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 6.
I20260709 12:18:04.438499 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.431269 18298 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:04.440613 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.446965 18373 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.451332 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.355055 21946 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.441273 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.455102 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.462368 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.462982 19040 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:04.464632 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.464974 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.465749 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.470937 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.473361 19040 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:04.476047 18367 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:04.476800 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.478433 19043 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:04.469756 21735 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:04.479880 18440 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:04.416261 21884 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:04.423125 21667 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.485843 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.487097 19043 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:04.488011 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:04.488265 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:04.488893 18297 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:04.475114 18196 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 3 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 8 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:04.481211 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.476747 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.490972 18369 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.491067 18299 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:04.482812 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.483781 18293 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: [6.5-7.6]
I20260709 12:18:04.480829 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:04.493095 18298 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:04.488096 18373 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:04.482111 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.494405 19036 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:04.427783 22064 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 12:18:04.469982 21921 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.454393 21968 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:18:04.487066 21417 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:04.561846 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.561290 18196 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:04.475666 21931 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:04.484277 21884 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:04.538559 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.539149 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.567639 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.568325 18367 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:18:04.545979 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54835.2us
I20260709 12:18:04.571331 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.572110 19040 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 5.
I20260709 12:18:04.545104 19036 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:04.564982 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.565428 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.487260 21735 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:04.484082 21856 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:18:04.514876 21729 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.561609 21968 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:04.565721 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.594600 18372 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:04.577934 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:04.539999 22066 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:18:04.580976 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.595085 18367 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:04.595448 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
W20260709 12:18:04.584354 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.535808 21921 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.596544 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.597262 18370 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.576853 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.590727 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:04.565114 21417 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.522792 21929 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:18:04.626719 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2292) took 1122 ms. Trace:
I20260709 12:18:04.628993 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.629544 19038 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.632400 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.498885 21945 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:04.608285 22041 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.142s	user 0.008s	sys 0.002s
I20260709 12:18:04.645967 18373 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:04.486845 22059 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:18:04.612491 21983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67a1783f49244bd5ba98721205d1ff6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:04.562232 21900 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:18:04.612035 21884 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.593976 21968 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:04.614058 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.614961 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.615682 22039 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.570780 22029 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.619244 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.620565 21735 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.576527 21347 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:04.623504 21975 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.602264 21729 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.566965 21931 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:04.627074 18443 rpcz_store.cc:276] 0709 12:18:03.488904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.560894 (+ 71990us) service_pool.cc:224] Handling call
0709 12:18:04.611108 (+1050214us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54000d8c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.611202 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087539968}
I20260709 12:18:04.653546 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:04.626220 21417 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:04.655321 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 641) took 1781 ms. Trace:
I20260709 12:18:04.591454 22078 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.686236 19038 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:04.605127 21921 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.513950 22084 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.158s
W20260709 12:18:04.656309 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 79534.9us
I20260709 12:18:04.657298 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.658082 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:04.692256 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.660247 18299 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:04.660809 21975 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.545316 21925 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:18:04.683336 18296 rpcz_store.cc:276] 0709 12:18:02.874293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.904502 (+ 30209us) service_pool.cc:224] Handling call
0709 12:18:03.986119 (+1081617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.986751 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.655108 (+668357us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002760d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:04.655262 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486741888}
I20260709 12:18:04.696161 19040 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:04.694365 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:04.688871 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98261.5us
I20260709 12:18:04.689685 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:04.692247 18372 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:04.697412 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:04.704370 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.661438 21347 consensus_queue.cc:1243] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:04.710180 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.685230 21884 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.732419 21968 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.682512 21931 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.660902 21947 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:18:04.705407 21798 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.700404 21564 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:04.713627 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:04.717207 21729 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:04.700637 18196 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 3 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 7 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } }
I20260709 12:18:04.740927 19038 raft_consensus.cc:606] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.652419 22043 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:04.717469 21938 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:18:04.727339 21642 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:04.727972 22050 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:04.719895 22039 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.743351 18369 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:04.741326 18196 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:04.747663 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 184813us
I20260709 12:18:04.758176 18375 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
W20260709 12:18:04.754374 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 185125us
I20260709 12:18:04.759554 18369 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.759693 18441 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.751426 18443 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:04.659410 21856 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:04.760380 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.762938 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.752907 19037 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:04.764276 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:04.696354 22029 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.697978 21975 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.701758 22052 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.719693 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:04.760385 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.769866 18372 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.770117 18442 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:04.760564 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.749423 18370 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:04.745241 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:04.746402 22084 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:04.775076 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:04.745309 21644 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.757988 22039 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.776006 18442 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:04.771642 18370 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:18:04.758666 21568 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.767843 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.752224 21945 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.758733 21925 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:04.766489 18445 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:04.755019 21635 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.779016 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.779405 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:04.793078 19037 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:04.781781 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.772044 22091 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.752491 21926 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:18:04.770788 21900 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.767001 21928 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.797736 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:04.801939 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:04.805088 19037 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:04.805431 19037 raft_consensus.cc:614] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:04.767527 22050 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:04.774541 22052 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.802855 19036 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:04.773066 21798 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:04.781062 22039 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.781878 21984 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:04.826076 18196 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 5, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:04.957074 18372 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:04.804548 21642 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.972633 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:04.974848 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:04.976141 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:04.819427 21884 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.161s
I20260709 12:18:04.981559 22043 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.981776 21417 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:04.980232 22088 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:18:05.007332 21798 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.010314 21798 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:04.803686 21900 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:04.807282 22029 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.825337 22052 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.806710 21635 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:04.886240 22042 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:04.794598 21568 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:04.798053 21945 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:04.971767 21958 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.003s
I20260709 12:18:04.805979 21942 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.208s
I20260709 12:18:05.013995 22091 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.802007 21925 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.028829 22078 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:05.047076 21916 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.750s	user 0.008s	sys 0.002s
I20260709 12:18:05.123890 18368 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:05.033142 21642 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:05.011942 21900 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:05.178094 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 396349us
I20260709 12:18:05.179132 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.138242 22095 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.121652 21798 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:05.204967 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 459752us
I20260709 12:18:04.991212 21931 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:05.216595 18367 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
I20260709 12:18:05.220588 18367 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:05.205964 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:05.223656 18196 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 6 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.207926 22095 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.205796 22091 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:05.133960 21735 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
W20260709 12:18:05.246733 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.248656 18196 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 5 to 6, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 6 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.212534 21798 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.241931 21926 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:18:05.227455 22057 raft_consensus.cc:2764] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:05.177726 21916 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.250723 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:05.255102 18300 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:05.260022 18196 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.241067 22029 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:05.267208 18196 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 3 to 4, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 4 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.247169 22095 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:05.269121 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 471421us
I20260709 12:18:05.253674 21947 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.254902 21916 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.252841 21798 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.256145 21417 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:05.268844 22097 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:18:05.278378 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:05.283623 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.021s	sys 0.022s
I20260709 12:18:05.285619 18447 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:05.286136 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:05.222507 22050 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:05.294224 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.021s	sys 0.022s
I20260709 12:18:05.286475 18445 pending_rounds.cc:85] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:18:05.286316 19037 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:05.253322 22091 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:05.249238 22103 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:18:05.299104 18445 pending_rounds.cc:107] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:18:05.303018 18445 raft_consensus.cc:2889] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:05.300468 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.298650 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 538115us
I20260709 12:18:05.276037 21644 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:05.277535 22029 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:05.300072 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:05.305563 18415 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.281129 21947 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:05.297672 21958 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:05.304955 21946 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.002s
W20260709 12:18:05.316300 21974 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.919s	user 0.012s	sys 0.003s
I20260709 12:18:05.255412 21635 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.269285 21642 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:05.319530 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.035s	sys 0.047s
I20260709 12:18:05.326334 18292 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:05.320500 21958 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:05.334824 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.035s	sys 0.047s
W20260709 12:18:05.347175 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 579742us
I20260709 12:18:05.348248 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:05.335145 18441 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:18:05.349255 18441 pending_rounds.cc:85] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 8
I20260709 12:18:05.349593 18441 pending_rounds.cc:107] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:05.349908 18441 raft_consensus.cc:2889] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:05.352681 18296 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:05.354497 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.355952 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.359347 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.330312 21945 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.360714 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.363124 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.334656 22106 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:05.368801 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.353495 22029 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.375463 22078 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:05.376907 22114 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.377789 21568 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.351042 21958 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.377643 22114 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.380399 21934 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:05.381006 21934 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:05.361995 21974 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:05.384337 21923 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.385011 21722 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:05.384928 21923 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.385496 21722 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:05.386456 21722 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.387341 21923 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.390777 22101 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.391376 22096 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:05.391608 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:05.426928 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 121429us
W20260709 12:18:05.393348 22098 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:18:05.393546 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:05.434129 18447 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:05.394295 22100 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:05.396405 22084 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.395813 22091 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.396579 21644 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:05.401576 22072 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.166s	user 0.007s	sys 0.008s
I20260709 12:18:05.402557 22109 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.403532 21931 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.404142 22103 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:05.405375 22099 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:18:05.407819 21921 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.708s	user 0.005s	sys 0.005s
I20260709 12:18:05.408284 22052 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.355825 21925 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.409731 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.459802 18298 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:05.412061 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 731) took 1345 ms. Trace:
I20260709 12:18:05.413064 21667 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:05.416862 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.417658 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:05.464593 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 46996.3us
I20260709 12:18:05.356433 22102 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.392223 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:05.431103 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:05.469749 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:05.434870 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:05.414657 22039 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.373835 21417 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:05.448385 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 148372us
I20260709 12:18:05.451496 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.418824 21926 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:05.450861 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 164285us
I20260709 12:18:05.568383 18448 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:05.424996 22101 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:05.573084 22101 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.460559 18374 rpcz_store.cc:276] 0709 12:18:04.066811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.066948 (+   137us) service_pool.cc:224] Handling call
0709 12:18:04.307906 (+240958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.310085 (+  2179us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:04.310106 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:04.310135 (+    29us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:04.310174 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.310217 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.337092 (+1026875us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18003ce800. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4
0709 12:18:05.411820 (+ 74728us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400168ad0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:05.411989 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1848,"raft.queue_time_us":1992,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7200,"spinlock_wait_cycles":557348480}
I20260709 12:18:05.461009 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:05.579895 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 118958us
I20260709 12:18:05.582444 22064 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.583092 22064 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.582278 22101 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.583379 21577 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:05.583571 22064 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.583936 21577 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.586198 22101 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.586635 21577 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.587412 21945 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:05.465487 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.589294 19037 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 7: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:05.466024 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:05.465495 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:05.500391 22102 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:05.483805 21925 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:05.495617 18295 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:05.496691 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1776) took 1085 ms. Trace:
I20260709 12:18:05.601173 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:05.543906 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 74257.1us
I20260709 12:18:05.601612 18368 rpcz_store.cc:276] 0709 12:18:04.410795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.429992 (+ 19197us) service_pool.cc:224] Handling call
0709 12:18:05.496445 (+1066453us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400395b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:05.496612 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592770944}
I20260709 12:18:05.461331 22109 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.606215 22109 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.467767 22103 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.608567 22103 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.607872 22102 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.612212 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:05.613102 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.468063 21921 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:05.467329 21975 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.003s
I20260709 12:18:05.502058 22114 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.629033 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.516152 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2226) took 1028 ms. Trace:
I20260709 12:18:05.469892 21946 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:05.418956 22106 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.372888 21947 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.494048 21856 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:05.496800 22052 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.554020 18446 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:05.558869 18447 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:05.515012 22084 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.649019 22084 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.572221 21926 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.566277 18443 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:05.588038 21945 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.596695 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:05.559458 22104 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:05.603417 18368 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:05.617745 21916 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.680253 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 739) took 1049 ms. Trace:
I20260709 12:18:05.680749 18368 rpcz_store.cc:276] 0709 12:18:04.630850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.602834 (+971984us) service_pool.cc:224] Handling call
0709 12:18:05.603192 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.679867 (+ 76675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.680183 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.604080 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 719) took 2343 ms. Trace:
I20260709 12:18:05.681869 18371 rpcz_store.cc:276] 0709 12:18:03.261026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.652167 (+391141us) service_pool.cc:224] Handling call
0709 12:18:04.143718 (+491551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.144393 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.603916 (+1459523us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400167bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:05.604027 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129741184}
I20260709 12:18:05.684262 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.523066 21974 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.618364 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.631796 18292 rpcz_store.cc:276] 0709 12:18:04.487576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.487760 (+   184us) service_pool.cc:224] Handling call
0709 12:18:05.515928 (+1028168us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54000726d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:05.516080 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926039936}
I20260709 12:18:05.690881 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 740) took 1059 ms. Trace:
I20260709 12:18:05.587160 21577 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.633015 22123 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:05.647972 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.648064 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:05.707336 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.707177 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.641921 21946 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:05.653781 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1928) took 1061 ms. Trace:
I20260709 12:18:05.618543 21925 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.686743 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:05.691402 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 73126.1us
I20260709 12:18:05.730640 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.691299 18368 rpcz_store.cc:276] 0709 12:18:04.631141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.689690 (+1058549us) service_pool.cc:224] Handling call
0709 12:18:05.690052 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.690293 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.690330 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.690387 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:05.690440 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.690716 (+   276us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.690832 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.566228 21417 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.645386 18440 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:05.702440 21577 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.736044 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:05.736342 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.737282 21577 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.703845 22106 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.704180 22105 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:05.648447 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:05.721145 18367 rpcz_store.cc:276] 0709 12:18:04.591902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.225457 (+633555us) service_pool.cc:224] Handling call
0709 12:18:05.225816 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.226123 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:05.226480 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.653392 (+426912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.653456 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:05.653518 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.653598 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.653716 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":1736,"raft.queue_time_us":101,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":10402,"thread_start_us":426449,"threads_started":1}
I20260709 12:18:05.634531 21923 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.738523 18440 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 6: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:05.738629 22052 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:05.738967 21923 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:05.726142 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.727442 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 741) took 1096 ms. Trace:
I20260709 12:18:05.739590 21923 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:05.739825 18371 rpcz_store.cc:276] 0709 12:18:04.631431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.689614 (+1058183us) service_pool.cc:224] Handling call
0709 12:18:05.726759 (+ 37145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.726997 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.727035 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.727091 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:05.727141 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.727209 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.727371 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.741480 18371 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:05.728783 18446 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:05.679477 22078 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.728793 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.742426 22078 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.743122 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:05.738453 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90111.8us
I20260709 12:18:05.744154 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.744313 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:05.745380 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:05.744537 22078 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.745550 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 742) took 1113 ms. Trace:
I20260709 12:18:05.737463 21568 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:05.746184 18371 rpcz_store.cc:276] 0709 12:18:04.631707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.740864 (+1109157us) service_pool.cc:224] Handling call
0709 12:18:05.741143 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.741776 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.745497 (+  3721us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.748473 18367 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:05.748533 18371 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:05.703192 21926 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.739364 22052 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:05.751299 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.750962 22052 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.752022 21926 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.752488 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:05.753135 21926 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.627610 21975 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:05.756502 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69878.3us
I20260709 12:18:05.765453 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.767495 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.742122 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 743) took 1110 ms. Trace:
I20260709 12:18:05.739591 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.769493 18368 rpcz_store.cc:276] 0709 12:18:04.631994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.741438 (+1109444us) service_pool.cc:224] Handling call
0709 12:18:05.741688 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.741832 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.741854 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.741890 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:05.741923 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.741984 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.742074 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:05.644901 22103 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 12:18:05.771849 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.773260 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 737) took 1143 ms. Trace:
I20260709 12:18:05.775902 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:05.776846 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:05.777613 18367 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:05.781041 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:05.782019 18368 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:05.788280 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47450.5us
W20260709 12:18:05.789032 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 323578us
I20260709 12:18:05.773624 18366 rpcz_store.cc:276] 0709 12:18:04.630236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.513459 (+883223us) service_pool.cc:224] Handling call
0709 12:18:05.513725 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.513912 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:05.514176 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.772955 (+258779us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.773016 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:05.773056 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.773113 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.773212 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":527,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":242568,"threads_started":1}
I20260709 12:18:05.794292 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:05.790903 21926 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.795615 21926 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.797415 21968 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:05.791167 22078 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.799286 22078 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:05.801810 22138 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:18:05.802934 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 736) took 1173 ms. Trace:
W20260709 12:18:05.803097 22119 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 12:18:05.804132 18440 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:05.803293 18372 rpcz_store.cc:276] 0709 12:18:04.629628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.510395 (+880767us) service_pool.cc:224] Handling call
0709 12:18:05.510823 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.526116 (+ 15293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.526161 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:05.561359 (+ 35198us) log.cc:844] Serialized 26 byte log entry
0709 12:18:05.762309 (+200950us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:05.762365 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.762455 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.801297 (+ 38842us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:05.802653 (+  1356us) raft_consensus.cc:1685] finished
0709 12:18:05.802671 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.802883 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3649,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":92568448,"thread_start_us":145491,"threads_started":1,"wal-append.queue_time_us":144365}
I20260709 12:18:05.697384 21916 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:05.805081 22129 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 12:18:05.806216 22029 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.806607 21916 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.806689 22029 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.801931 22078 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.808822 22029 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.810334 19040 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.811872 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.820665 19037 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:05.828293 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:05.770922 22061 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.006s
W20260709 12:18:05.830538 22061 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.007s
I20260709 12:18:05.844745 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:05.847332 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.847595 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:05.848795 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:05.849530 19037 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:05.852092 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:05.808363 21926 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:05.850852 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:05.791025 22106 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.832894 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:05.787981 22105 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:05.725819 22096 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:18:05.721246 22120 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
W20260709 12:18:05.835083 22095 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 12:18:05.835759 22091 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.837888 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.969291 18444 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
W20260709 12:18:05.970122 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.841531 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:05.814105 22122 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.722069 22123 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.873034 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:05.922252 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 176940us
W20260709 12:18:05.983246 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.814287 21974 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:05.843989 21916 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:05.844100 21931 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:05.974720 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119700us
W20260709 12:18:05.975303 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 124517us
I20260709 12:18:05.841022 22104 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:05.844713 21957 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.843796 21577 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 12:18:05.968374 22102 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:05.845088 22039 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.005878 21577 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:06.006296 21577 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:06.006860 22093 raft_consensus.cc:993] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:18:06.007151 22039 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.007422 22093 raft_consensus.cc:1081] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:18:06.007858 22039 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.010317 22039 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.007192 21577 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.012764 21577 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:05.903549 22084 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:05.984985 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:05.900714 21648 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:18:05.857834 22121 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.842574 22108 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 12:18:05.969509 22078 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.011387 21934 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.058454 21934 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.058282 22078 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:05.998077 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:06.060770 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 62811.5us
I20260709 12:18:05.974367 21729 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:05.844396 21856 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:05.985399 21958 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.005s
I20260709 12:18:05.969761 22029 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:05.969218 21635 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:05.965922 21946 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.027660 22050 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.028180 22084 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.004290 22103 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.033143 21945 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.013932 22104 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.001371 21974 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.013993 22102 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.038255 21923 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.021171 22052 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.058022 22064 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:06.044185 21667 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.003s
I20260709 12:18:05.968413 22095 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.045594 22137 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:06.051970 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67081us
W20260709 12:18:06.056159 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 204169us
I20260709 12:18:06.101714 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:06.050220 22105 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.970306 21564 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:18:05.995653 21931 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:06.071072 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.104351 22121 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.105315 22137 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:06.147890 22121 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.149991 22121 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.105861 18375 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:06.106271 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 1911) took 1575 ms. Trace:
I20260709 12:18:06.151969 18373 rpcz_store.cc:276] 0709 12:18:04.530806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.536417 (+  5611us) service_pool.cc:224] Handling call
0709 12:18:06.106065 (+1569648us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b54000fa354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:06.106212 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177570176}
W20260709 12:18:06.107789 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.154134 21667 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.154392 21974 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:06.154951 21974 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.154846 21667 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.155560 21974 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.157600 21667 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.002962 21957 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.158308 21957 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.070668 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:06.160480 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:06.162009 18373 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:06.148656 22137 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:05.968775 22091 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.163061 22137 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:06.163313 22114 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.111328 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.077591 18449 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:06.078188 18370 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:06.079790 18366 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:18:06.064826 22108 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:06.066123 21729 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:06.075925 21577 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:06.078802 21926 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.079197 21921 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:06.043836 22132 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:06.104283 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.076406 21856 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.097350 21417 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.089692 22050 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.079496 22029 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.094058 22103 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.103992 21642 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.157737 21974 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.022074 21568 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:06.113466 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 815) took 1351 ms. Trace:
I20260709 12:18:06.076231 21958 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.163831 22114 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.107276 19040 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:06.373366 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:06.261873 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 823) took 1487 ms. Trace:
I20260709 12:18:06.147279 21931 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:06.099546 22102 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:06.415944 18442 rpcz_store.cc:276] 0709 12:18:04.778390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.778843 (+   453us) service_pool.cc:224] Handling call
0709 12:18:04.783962 (+  5119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.784216 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:05.258493 (+474277us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.425119 (+166626us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.607080 (+181961us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b18003e92c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:06.121913 (+514833us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b18003e92c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:18:06.122111 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 12:18:06.261481 (+139370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:06.261549 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:06.261605 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.261686 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:06.261807 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":865336,"prepare.queue_time_us":886029,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":106,"raft.queue_time_us":809,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7060,"spinlock_wait_cycles":28318848,"thread_start_us":468103,"threads_started":2,"wal-append.queue_time_us":316754}
W20260709 12:18:06.297470 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.437568 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.410616 18441 rpcz_store.cc:276] 0709 12:18:04.762002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.763807 (+  1805us) service_pool.cc:224] Handling call
0709 12:18:04.764063 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.350580 (+586517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.113174 (+762594us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002030d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:06.113393 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307252608}
I20260709 12:18:06.410028 21925 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:06.252832 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.220465 18371 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.125375 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 738) took 1494 ms. Trace:
I20260709 12:18:06.443550 18374 rpcz_store.cc:276] 0709 12:18:04.630551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.579891 (+949340us) service_pool.cc:224] Handling call
0709 12:18:05.580192 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.651922 (+ 71730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.651950 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:05.652013 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:05.652044 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.125082 (+473038us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b54003f2d54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:06.125145 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:06.125297 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164529920}
I20260709 12:18:06.444418 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.445007 18292 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:06.445191 18296 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:06.217206 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51940us
I20260709 12:18:06.447513 22146 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.448266 22146 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.451061 22148 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.451766 22148 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.454643 18440 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:06.459337 22150 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.299s	user 0.007s	sys 0.025s
I20260709 12:18:06.424993 21577 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.462162 21577 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.476358 18374 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:06.322499 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:06.221374 21975 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.097023 21916 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:06.480847 21916 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.098847 22084 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:06.483593 22084 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.486303 22084 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.427641 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:06.487121 18373 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:06.109442 21900 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.494292 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:06.145467 22095 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.165197 22101 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.496284 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.499123 18296 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:06.217420 21667 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.147084 21564 consensus_queue.cc:1243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:06.166254 21635 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.498447 18370 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:06.501004 22078 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.143680 21923 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.141014 21945 raft_consensus.cc:2764] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:06.504866 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:06.497078 21983 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.846s	user 0.002s	sys 0.008s
I20260709 12:18:06.500876 21729 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:06.523883 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.017s	sys 0.014s
I20260709 12:18:06.433650 22057 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:06.434217 22091 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.512415 22064 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:06.532389 18263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.024s	sys 0.011s
I20260709 12:18:06.516201 21957 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.520352 18296 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:06.520023 21635 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.520303 18440 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:06.526079 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.526271 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:06.524358 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.017s	sys 0.014s
I20260709 12:18:06.257974 22039 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.245s
I20260709 12:18:06.528520 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.550056 18298 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:06.495348 22144 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.514048 22104 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.105178 22052 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.532953 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:06.533375 18263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.024s	sys 0.011s
I20260709 12:18:06.504031 21975 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.536275 18291 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.537892 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.544010 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.559619 19037 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:06.559850 18292 raft_consensus.cc:2410] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:06.498406 21417 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.423496 21926 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.536938 21856 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.433462 21642 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.587720 18372 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:06.553205 18371 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:18:06.553514 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.529001 22096 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.555552 18293 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.524334 22122 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:06.557792 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.562462 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.532070 22078 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.591727 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 1.90547e+06us
I20260709 12:18:06.596105 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:06.597666 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:06.598564 21921 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.522365 21667 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.599325 21921 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.601378 21667 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.602735 18367 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:18:06.603147 18367 pending_rounds.cc:85] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:18:06.603423 18367 pending_rounds.cc:107] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:18:06.603696 18367 raft_consensus.cc:2889] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:06.598999 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:06.606940 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49239.6us
I20260709 12:18:06.521736 22101 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.607630 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:06.553848 21945 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.619560 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2318) took 1056 ms. Trace:
I20260709 12:18:06.633348 18443 rpcz_store.cc:276] 0709 12:18:05.562903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.745017 (+182114us) service_pool.cc:224] Handling call
0709 12:18:05.745301 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.850677 (+105376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.850712 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:05.851301 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:18:06.256929 (+405628us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:06.257095 (+   166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.257186 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:06.619334 (+362148us) raft_consensus.cc:1685] finished
0709 12:18:06.619356 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:06.619502 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104994,"raft.queue_time_us":105880,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":17816,"spinlock_wait_cycles":12672,"thread_start_us":263251,"threads_started":1,"wal-append.queue_time_us":548579}
I20260709 12:18:06.559006 22144 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.621726 22103 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:06.623814 21957 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.552381 22123 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:06.630584 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.638979 22033 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:06.692463 22033 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:06.565515 21729 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:06.641619 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.664361 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.586766 22104 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:06.683779 22126 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.949s	user 0.005s	sys 0.000s
W20260709 12:18:06.684319 22152 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.577s	user 0.013s	sys 0.012s
I20260709 12:18:06.708683 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.587114 22145 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.691900 21926 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.693794 21642 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.535058 22057 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.697074 19036 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:06.700214 22052 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.712370 19037 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:18:06.557617 22071 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 12:18:06.689807 22149 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:06.721618 19038 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:06.724289 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:06.586724 21975 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.592702 21417 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.729089 18293 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:06.730381 18291 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:18:06.759184 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.730630 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:06.760406 18292 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.686291 21856 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.734514 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:06.774376 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 175450us
I20260709 12:18:06.775636 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:06.776254 19039 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.776327 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.631855 22078 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.820488 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.821270 18293 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:06.679494 21945 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.625356 22158 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:18:06.726517 22043 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.627425 22096 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.777858 21921 raft_consensus.cc:2764] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:06.842262 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:06.846112 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.713927 22104 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.821687 22149 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.724327 22145 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:06.836925 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.822424 21856 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:06.861723 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2231) took 1508 ms. Trace:
I20260709 12:18:06.863725 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:06.864774 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.866925 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.827636 21983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59422 (request call id 647) took 5200 ms. Trace:
I20260709 12:18:06.837301 21818 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:06.829646 21958 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.871307 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.872749 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.876852 18445 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:06.831686 21974 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.877240 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:06.877867 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:06.878580 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.879514 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.879845 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.781776 21923 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:18:06.874372 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.885825 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:06.862151 18299 rpcz_store.cc:276] 0709 12:18:05.353480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.353603 (+   123us) service_pool.cc:224] Handling call
0709 12:18:05.353890 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.354202 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.354219 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:05.374201 (+ 19982us) log.cc:844] Serialized 26 byte log entry
0709 12:18:06.073716 (+699515us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b180006da80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:06.106310 (+ 32594us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:06.106345 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.106409 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:06.683385 (+576976us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:06.861522 (+178137us) raft_consensus.cc:1685] finished
0709 12:18:06.861546 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:06.861666 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2075,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":14758,"spinlock_wait_cycles":122070272,"thread_start_us":32410,"threads_started":1,"wal-append.queue_time_us":697144}
I20260709 12:18:06.847743 22043 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.888923 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.853969 22050 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.890877 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.893152 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.895890 19044 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:06.895084 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:06.895870 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.897143 18442 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:06.897917 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:06.898720 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.900146 18443 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:06.903074 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.875370 21957 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.900885 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:06.910480 22144 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.875903 21818 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:06.912446 21417 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.912559 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:06.913384 18443 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:06.913178 21856 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:06.916550 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.917277 22078 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.890050 21974 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.931160 22103 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:06.820482 22033 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.940918 22103 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.942116 21667 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:06.942593 21667 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.942947 21667 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:06.901724 22160 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:06.945461 22160 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:06.946563 21975 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.929222 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 821) took 2155 ms. Trace:
I20260709 12:18:06.947399 18448 rpcz_store.cc:276] 0709 12:18:04.773386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.773492 (+   106us) service_pool.cc:224] Handling call
0709 12:18:04.773789 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.568805 (+795016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.928895 (+1360090us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54002a9c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:06.929149 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826775296}
I20260709 12:18:06.926419 19039 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:06.936156 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.950431 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:06.873256 21983 rpcz_store.cc:276] 0709 12:18:01.586459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.593901 (+  7442us) service_pool.cc:224] Handling call
0709 12:18:02.383387 (+789486us) ts_tablet_manager.cc:934] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
0709 12:18:04.484568 (+2101181us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:05.650511 (+1165943us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:05.650628 (+   117us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:06.547458 (+896830us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:06.631753 (+ 84295us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:06.775544 (+143791us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:06.781208 (+  5664us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":40330,"tablet-copy.queue_time_us":380584,"thread_start_us":323857,"threads_started":2}
I20260709 12:18:06.931975 22123 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:06.952005 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.823436 22151 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
W20260709 12:18:06.934823 22137 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.771s	user 0.003s	sys 0.004s
I20260709 12:18:06.955506 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.954895 19038 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 7: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:06.948664 21945 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.962159 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:06.972777 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:06.960932 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65927.4us
I20260709 12:18:06.973997 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.964387 22078 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:06.964797 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0931a54c049145dfad672173c0b29a40"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:06.979512 19044 raft_consensus.cc:606] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:06.976011 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.965659 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1919) took 2420 ms. Trace:
I20260709 12:18:06.967433 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.980587 19043 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:06.980588 18375 rpcz_store.cc:276] 0709 12:18:04.545447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.759024 (+213577us) service_pool.cc:224] Handling call
0709 12:18:04.759303 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.151397 (+1392094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.965382 (+813985us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400395154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:06.965585 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095803776}
I20260709 12:18:06.968381 19038 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:06.958549 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:06.923826 22131 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.759s	user 0.003s	sys 0.000s
I20260709 12:18:06.922228 22039 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.974431 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.923858 22057 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:06.989929 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:06.832206 22122 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:06.993273 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 217013us
I20260709 12:18:06.927547 22146 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.956491 21923 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:06.915004 21931 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:06.995543 21983 tablet_copy_client.cc:806] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:18:06.990842 21945 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.986603 22103 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.998507 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:06.956624 22123 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:06.999397 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:06.999903 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.000416 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:07.054617 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54299.2us
I20260709 12:18:06.964952 21958 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:07.003135 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:06.976797 22158 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.985349 21926 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:06.930198 22104 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:06.987785 22064 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:07.009143 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2225) took 2518 ms. Trace:
I20260709 12:18:07.072690 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.072497 18300 rpcz_store.cc:276] 0709 12:18:04.485201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.485751 (+   550us) service_pool.cc:224] Handling call
0709 12:18:04.523528 (+ 37777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.260164 (+736636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.003573 (+1743409us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b540004db54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:07.003738 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682024704}
I20260709 12:18:06.999553 22170 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:07.012799 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 120695us
I20260709 12:18:06.987560 22071 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:07.052675 21929 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:06.993173 22166 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:06.999524 22091 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:07.061494 18298 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:07.014199 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:07.083325 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.067858 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:07.084223 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.084529 18443 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:07.068032 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64980.3us
I20260709 12:18:07.056646 21923 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:07.073873 18440 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:07.061926 22078 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.052125 21983 tablet_copy_client.cc:670] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:18:07.086169 21729 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:18:07.074525 22158 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:07.099630 18448 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:07.007782 21934 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:07.007341 21642 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:06.993643 22155 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:18:07.014017 21945 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.088089 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.088188 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:07.088471 22043 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.088829 22144 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:07.073127 22137 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:07.069891 22064 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.090061 21931 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:07.091661 21577 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 12:18:07.093154 21957 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:07.072981 21958 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.063736 21926 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.094192 22145 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.094831 22096 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:07.101326 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.083343 22071 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.085958 22168 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:07.110073 18292 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
W20260709 12:18:07.088117 22163 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:18:07.199839 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.201670 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:07.080538 22114 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:07.200841 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80240.3us
I20260709 12:18:07.243619 18443 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.246557 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:07.247318 19038 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.250828 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:07.251636 19039 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:07.254206 18443 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:07.207230 21635 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.001s
I20260709 12:18:07.255576 21635 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.215433 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:07.256593 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:07.256709 21958 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.257694 22168 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.258888 21958 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.121918 22104 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:07.260772 22104 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.262230 21722 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.262636 22178 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:07.214692 22102 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.718s	user 0.005s	sys 0.005s
I20260709 12:18:07.263195 22178 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.263350 22102 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.101168 21929 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.224776 18440 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:07.100215 22166 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:07.225163 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 137060us
W20260709 12:18:07.226418 18295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2228) took 1888 ms (client timeout 2417 ms). Trace:
I20260709 12:18:07.100172 22170 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.121810 21642 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.221324 22158 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:07.213662 22120 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:18:07.230396 22091 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.088748 22123 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.225286 22078 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.231480 22171 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.230823 21945 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.242604 22071 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.242331 21926 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.259824 22168 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:07.259971 22050 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:07.262774 22104 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:07.210371 21934 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.264986 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.392743 18448 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:07.382623 21983 tablet_copy_client.cc:538] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:18:07.274320 19038 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:07.238240 22137 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:07.379317 22158 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.231266 21923 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.240057 21577 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.370205 21729 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.372116 22146 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.360689 22043 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:07.236603 22064 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:07.409555 22146 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.410490 21929 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.395386 21934 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.380635 19044 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:07.382846 22120 consensus_queue.cc:1243] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:07.390697 22123 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.389438 22178 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.382189 22172 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:18:07.383579 22091 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.387498 22078 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.418674 22146 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:07.287169 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:07.363477 18295 rpcz_store.cc:269] 0709 12:18:05.337601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.337758 (+   157us) service_pool.cc:224] Handling call
0709 12:18:07.226166 (+1888408us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400090454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:07.226348 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274190208}
W20260709 12:18:07.368978 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 313609us
I20260709 12:18:07.283514 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:07.332113 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.378461 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:07.265365 18441 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:07.375433 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2332) took 1567 ms. Trace:
I20260709 12:18:07.419375 21635 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.820251 21958 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.674284 21945 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.821477 21958 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.822660 21635 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:07.470259 21667 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.005s
I20260709 12:18:07.823648 21947 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.824469 21958 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.824559 21667 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.408380 21729 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.486232 21916 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:07.828086 21916 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:07.825218 21642 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:07.822772 21577 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.778637 18442 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:07.826929 21958 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.818423 18448 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:18:07.770838 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.834s (SENT) [suppressed 11 similar messages]
I20260709 12:18:07.799191 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:07.829608 22122 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.911821 21729 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:07.478134 22127 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.746s	user 0.004s	sys 0.019s
I20260709 12:18:07.916254 21934 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.917480 22104 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.470824 22137 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:07.429096 22164 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.543s	user 0.002s	sys 0.000s
W20260709 12:18:07.921290 21722 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.006s
I20260709 12:18:07.403808 22050 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:07.658668 22064 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.922363 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:07.416880 22043 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.923480 18449 rpcz_store.cc:276] 0709 12:18:05.808176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.841297 (+ 33121us) service_pool.cc:224] Handling call
0709 12:18:07.375263 (+1533966us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400422f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:07.375380 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236745600}
I20260709 12:18:07.925360 22091 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.684769 22071 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.430851 22101 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.921334 22078 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.928428 21945 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:07.649925 22105 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.497s	user 0.014s	sys 0.005s
I20260709 12:18:07.404625 22158 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.933704 22121 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:07.934042 22104 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.507587 22102 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.508522 22148 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.484031 22166 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.942273 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:07.498214 21974 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.505375 22170 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:07.942767 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 144044us
I20260709 12:18:07.950558 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:07.951426 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:07.951562 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:07.947997 22122 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.941975 19040 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
W20260709 12:18:07.925390 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 546812us
I20260709 12:18:07.506753 22029 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.943727 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.963078 19038 raft_consensus.cc:606] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:07.963470 21856 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:07.968894 21856 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.963627 22107 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.971500 21642 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:07.971664 22107 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.972030 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:07.973773 21818 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.945181 22121 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.975344 22121 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.975718 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:07.942646 21667 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:07.977221 22071 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:07.967485 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:07.968250 19042 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:07.938983 21635 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:07.906237 21916 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:07.987752 22033 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.068184 22190 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.069036 22190 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.069756 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:08.070585 18445 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:08.070782 22149 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.999332 18440 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:08.071530 22149 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.072147 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1939) took 1066 ms. Trace:
I20260709 12:18:08.072638 18443 rpcz_store.cc:276] 0709 12:18:07.005721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.254924 (+249203us) service_pool.cc:224] Handling call
0709 12:18:07.255145 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.301106 (+ 45961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:07.301170 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:07.301229 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:07.301289 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.071702 (+770413us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b54002594d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:08.071766 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.071980 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105117952}
I20260709 12:18:08.074137 18440 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:07.987939 21958 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:07.983863 19038 raft_consensus.cc:614] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:08.001039 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.008829 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:08.122782 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.078577 19037 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:08.095188 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 828) took 2525 ms. Trace:
I20260709 12:18:08.084674 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:07.995585 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1923) took 3415 ms. Trace:
I20260709 12:18:08.110484 18440 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:08.090219 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 672) took 1092 ms. Trace:
W20260709 12:18:08.216248 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 445738us
I20260709 12:18:08.094348 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1792) took 1659 ms. Trace:
W20260709 12:18:08.159721 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 208719us
I20260709 12:18:07.990046 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2246) took 1430 ms. Trace:
W20260709 12:18:08.141886 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.002s	sys 0.027s
I20260709 12:18:08.098975 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 926) took 1573 ms. Trace:
I20260709 12:18:08.088284 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1786) took 2240 ms. Trace:
I20260709 12:18:08.064710 21983 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
W20260709 12:18:08.257565 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.003s	sys 0.032s
I20260709 12:18:08.080034 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 830) took 2509 ms. Trace:
I20260709 12:18:08.109779 22102 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.198077 21926 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.805s	user 0.004s	sys 0.013s
I20260709 12:18:08.007364 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 827) took 2366 ms. Trace:
I20260709 12:18:08.248765 18367 rpcz_store.cc:276] 0709 12:18:06.434426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.434589 (+   163us) service_pool.cc:224] Handling call
0709 12:18:06.434931 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.604181 (+169250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.093842 (+1489661us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400395b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:08.094152 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384844032}
I20260709 12:18:08.106123 21818 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:08.093483 19043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 896) took 1369 ms (client timeout 1776 ms). Trace:
I20260709 12:18:08.077840 22149 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:08.077188 22105 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.793s	user 0.014s	sys 0.009s
I20260709 12:18:08.132808 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:08.247962 19044 rpcz_store.cc:276] 0709 12:18:06.998104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.998253 (+   149us) service_pool.cc:224] Handling call
0709 12:18:08.089835 (+1091582us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b5400611054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.090026 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752938368}
I20260709 12:18:08.253782 18372 rpcz_store.cc:276] 0709 12:18:05.847213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.847345 (+   132us) service_pool.cc:224] Handling call
0709 12:18:08.087796 (+2240451us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54003f1454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.088026 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700213120}
I20260709 12:18:08.251940 19036 rpcz_store.cc:276] 0709 12:18:06.525919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.526039 (+   120us) service_pool.cc:224] Handling call
0709 12:18:08.098625 (+1572586us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54005316d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.098811 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338474880}
I20260709 12:18:08.078306 21635 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:08.090960 19040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 925) took 1565 ms (client timeout 2033 ms). Trace:
I20260709 12:18:08.009199 22104 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.248356 18446 rpcz_store.cc:276] 0709 12:18:05.569680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.119707 (+550027us) service_pool.cc:224] Handling call
0709 12:18:06.120049 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.640375 (+520326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.640419 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:06.640477 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:06.640519 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.094695 (+1454176us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54002ab054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:08.094804 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.095022 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1196005248}
I20260709 12:18:08.125622 22043 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.239368 21916 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.997392 21945 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:07.989982 22122 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.249141 18298 rpcz_store.cc:276] 0709 12:18:06.559756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.562147 (+  2391us) service_pool.cc:224] Handling call
0709 12:18:07.989641 (+1427494us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540009ef54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:07.989830 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071252864}
I20260709 12:18:08.258546 18441 rpcz_store.cc:276] 0709 12:18:05.570290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.441009 (+870719us) service_pool.cc:224] Handling call
0709 12:18:06.479584 (+ 38575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.921877 (+1442293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.079699 (+157822us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400354dd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:08.079883 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806643584}
I20260709 12:18:08.001403 22050 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.080363 21722 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:07.989429 22064 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.075780 22190 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.143198 22052 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.988340 22107 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.260275 18444 rpcz_store.cc:276] 0709 12:18:05.569325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.115765 (+546440us) service_pool.cc:224] Handling call
0709 12:18:06.116137 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:06.498717 (+382580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.498746 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:06.498781 (+    35us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:06.498818 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.935212 (+1436394us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400423e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:07.935288 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.935534 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879373568}
W20260709 12:18:08.261183 19043 rpcz_store.cc:269] 0709 12:18:06.723750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.723937 (+   187us) service_pool.cc:224] Handling call
0709 12:18:08.093077 (+1369140us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.093282 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347852032}
I20260709 12:18:08.106092 21930 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.280699 21930 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.284046 21945 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:08.285554 21945 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.000859 22158 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.291720 22148 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.990401 21856 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.121961 22121 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.292363 22148 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.294445 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.294575 22121 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.295208 18368 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:08.268309 21926 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.296636 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.296985 21930 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.297248 21926 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.298887 18449 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:08.299759 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.302340 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:08.217589 22078 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.303480 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:08.304302 18370 raft_consensus.cc:2368] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10.
I20260709 12:18:08.303690 22078 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.305387 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.306250 18373 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:08.306437 21729 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.267907 22185 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:08.307192 22185 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.276063 22178 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.310436 22178 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.312071 22052 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.316983 22078 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.319829 22102 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.322139 22102 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.306977 21729 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.324175 21729 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.276204 21722 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.325664 21722 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:08.331353 22169 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.332026 18442 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:08.331974 22169 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:08.333596 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58740.3us
I20260709 12:18:08.335148 22169 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.065094 22091 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.271s
I20260709 12:18:08.155373 22071 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:08.093472 21648 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.180s
I20260709 12:18:08.249912 21934 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:08.268486 19040 rpcz_store.cc:269] 0709 12:18:06.525525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.525723 (+   198us) service_pool.cc:224] Handling call
0709 12:18:08.090746 (+1565023us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540048ec54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.090903 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3194461440}
I20260709 12:18:08.063589 22160 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:08.278271 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.260376 21577 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.291494 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:07.996654 21968 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:08.358366 21968 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:08.359771 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 227351us
I20260709 12:18:08.359663 21968 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.360723 21968 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.361449 21921 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.362074 21921 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:08.350394 22190 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.091697 22195 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.179s
I20260709 12:18:08.363590 22190 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:08.344249 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.366016 22190 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.353190 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.355607 22148 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:08.348199 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 983004us
I20260709 12:18:08.352497 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:08.366848 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:08.357245 21958 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.365715 21856 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:08.357884 18440 raft_consensus.cc:1217] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-7.6]   Dedup: 4.5->[7.6-7.6]
I20260709 12:18:08.370090 18440 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [4.5-7.6]
I20260709 12:18:08.350137 18371 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:08.362576 21921 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.374819 18194 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.377270 18194 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.368234 19044 raft_consensus.cc:606] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:08.379742 18374 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:08.379915 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 104259us
I20260709 12:18:08.384265 18295 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:08.389160 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:08.384997 18300 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:08.383277 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.398412 18299 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:08.383419 22149 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:08.399806 22149 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:08.401928 21930 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:08.402411 22149 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.402871 22195 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:08.383725 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.403328 22195 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:08.380237 22091 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.384897 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:08.404115 18293 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
W20260709 12:18:08.387857 18447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2325) took 2822 ms (client timeout 1834 ms). Trace:
W20260709 12:18:08.432169 18447 rpcz_store.cc:269] 0709 12:18:05.564879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.647718 (+ 82839us) service_pool.cc:224] Handling call
0709 12:18:08.387445 (+2739727us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:08.387620 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6112897792}
I20260709 12:18:08.419716 19044 raft_consensus.cc:606] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:08.433887 21958 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.379205 18194 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.385267 22173 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.437367 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.405754 22091 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.442883 21958 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.446166 18295 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:08.402539 21921 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.386905 22178 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.380088 22078 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.454450 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:08.380249 22071 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.471095 18292 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.404425 21577 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.247675 18369 rpcz_store.cc:276] 0709 12:18:04.580024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.761690 (+181666us) service_pool.cc:224] Handling call
0709 12:18:04.762185 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.134643 (+372458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:05.134673 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:05.176996 (+ 42323us) log.cc:844] Serialized 26 byte log entry
0709 12:18:05.335372 (+158376us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:05.420979 (+ 85607us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.421668 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.004042 (+1582374us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18003ceb60. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419
0709 12:18:07.994979 (+990937us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:07.995049 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:07.995125 (+    76us) raft_consensus.cc:1685] finished
0709 12:18:07.995137 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.995365 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85910,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":865,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":39945,"spinlock_wait_cycles":891575040,"thread_start_us":135795,"threads_started":1,"wal-append.queue_time_us":250318,"wal-append.run_cpu_time_us":1999,"wal-append.run_wall_time_us":1045175}
I20260709 12:18:08.454818 22091 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.473871 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.391407 21934 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.482825 18371 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:08.456521 22195 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.502341 22052 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.503822 18367 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:08.507405 22185 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.507845 22160 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.507898 18295 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:08.508566 22160 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.510377 21921 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.511713 19037 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:08.512327 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.513028 18291 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.514133 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.514328 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.511534 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.515720 18294 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.516211 22160 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.483332 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.518064 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.520320 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:08.521211 22185 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.521597 22185 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:08.522013 22185 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.521602 21983 tablet_bootstrap.cc:700] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent opening tablet: real 0.175s	user 0.005s	sys 0.000s
I20260709 12:18:08.528642 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.529692 21934 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.530469 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.530926 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.531440 18297 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:08.531597 18299 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.532529 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.532152 18194 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 5 to 9, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 9 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.533200 18300 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:08.533739 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.533982 18194 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.536142 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.536705 18373 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:08.537643 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:08.538335 18370 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:08.538277 18295 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:08.540103 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.532565 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:08.552428 18374 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:18:08.551923 18447 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:08.553287 18374 pending_rounds.cc:85] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 9
I20260709 12:18:08.556277 18369 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:08.562548 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.561880 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.558393 18374 pending_rounds.cc:107] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:08.569105 18374 raft_consensus.cc:2889] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:08.569948 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 254251us
I20260709 12:18:08.580256 18372 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:08.574510 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:08.580768 18372 pending_rounds.cc:85] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:08.586830 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.592849 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:08.587321 18371 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:08.562150 18194 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.595055 18295 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:08.598618 18372 pending_rounds.cc:107] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:18:08.598218 21631 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 12:18:08.605381 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62857.8us
I20260709 12:18:08.606098 18370 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 12:18:08.607760 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:08.598990 18372 raft_consensus.cc:2889] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:08.566213 22078 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.608377 19044 raft_consensus.cc:606] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:08.612717 19044 raft_consensus.cc:614] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:08.598778 21729 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:08.562495 22173 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.621860 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.624074 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:08.617756 21729 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.635849 22173 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.640146 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.640568 18293 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:08.640789 18300 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.629488 21958 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.630034 21945 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:08.634768 21729 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:08.659857 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.651513 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:08.671118 18300 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:08.677745 18194 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.688086 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.691417 18374 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:08.692761 22199 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:18:08.695287 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 295) took 25007 ms (client timeout 15000 ms). Trace:
I20260709 12:18:08.695415 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.695509 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 840) took 1439 ms. Trace:
I20260709 12:18:08.696199 18296 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:08.679416 21631 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.675663 21945 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:08.699536 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 337316us
I20260709 12:18:08.699622 18194 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:08.695982 18448 rpcz_store.cc:276] 0709 12:18:07.255818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.086494 (+830676us) service_pool.cc:224] Handling call
0709 12:18:08.087113 (+   619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:08.087811 (+   698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.087855 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:08.088544 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 12:18:08.407829 (+319285us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:08.407884 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.407962 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:08.695285 (+287323us) raft_consensus.cc:1685] finished
0709 12:18:08.695301 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.695439 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":318840,"threads_started":1,"wal-append.queue_time_us":502800}
I20260709 12:18:08.710757 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.712214 18300 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:08.717123 22198 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:18:08.700459 22148 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:18:08.725891 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.728901 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.735816 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.702469 21631 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.729046 22169 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:08.763837 18291 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:08.743840 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.779822 18292 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.726749 18299 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.782356 18294 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:08.782871 18194 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.785581 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:08.782467 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52984.8us
I20260709 12:18:08.783950 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 1952) took 1552 ms. Trace:
I20260709 12:18:08.736181 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:08.789676 18296 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:18:08.738057 22195 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:08.790803 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.738209 22071 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.779630 21631 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.776178 21577 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.787446 18443 rpcz_store.cc:276] 0709 12:18:07.231883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.074391 (+842508us) service_pool.cc:224] Handling call
0709 12:18:08.074883 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:08.205892 (+131009us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:08.565727 (+359835us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.783625 (+217898us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:08.783673 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:08.783736 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.783793 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.783890 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":430917,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"thread_start_us":443945,"threads_started":2}
I20260709 12:18:08.780567 22169 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:08.804765 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 224148us
I20260709 12:18:08.804838 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.807555 18297 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:08.809677 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.809983 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.810519 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:08.810948 18293 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:08.820672 18369 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:08.790031 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.815471 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.811452 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:08.822907 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:08.738945 22185 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.823086 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:08.821718 18374 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:08.825237 19044 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:08.829250 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.831545 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.832641 19035 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:18:08.834753 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:08.835503 18194 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 5 to 8, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:08.839359 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.839996 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.840979 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.841794 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:08.848853 22169 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.851720 21945 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.854100 19037 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:08.856179 22104 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:08.856287 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53041.3us
W20260709 12:18:08.856926 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 240435us
I20260709 12:18:08.867095 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:08.829463 18294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:43604
I20260709 12:18:08.861454 18291 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:08.867774 18294 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:08.857788 18295 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: [4.5-7.6]
W20260709 12:18:08.861912 22202 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 12:18:08.865537 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 64815.1us
W20260709 12:18:08.867326 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.868135 18294 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.862661 18297 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:08.872862 19040 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:08.877326 18372 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:08.878351 18366 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:08.797804 21722 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.879608 18292 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:08.695709 18196 rpcz_store.cc:269] 0709 12:17:43.601082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.634215 (+ 33133us) service_pool.cc:224] Handling call
0709 12:18:08.596529 (+24962314us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:05.844099 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:06.046767 (+202668us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:06.046800 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:18:06.046816 (+    16us) tablet.cc:662] Decoding operations
0709 12:18:06.047789 (+   973us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:06.047843 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:06.047869 (+    26us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:18:06.048618 (+   749us) tablet.cc:701] Row locks acquired
0709 12:18:06.048635 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:18:06.048766 (+   131us) write_op.cc:270] Start()
0709 12:18:06.048875 (+   109us) write_op.cc:276] Timestamp: P: 1783599486048721 usec, L: 0
0709 12:18:06.048897 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:18:06.049506 (+   609us) log.cc:844] Serialized 4010 byte log entry
0709 12:18:06.859503 (+809997us) op_driver.cc:464] REPLICATION: finished
0709 12:18:07.223735 (+364232us) write_op.cc:301] APPLY: starting
0709 12:18:07.230784 (+  7049us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:07.267894 (+ 37110us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:07.373736 (+105842us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:07.425281 (+ 51545us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:07.425649 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:18:07.425724 (+    75us) write_op.cc:312] APPLY: finished
0709 12:18:07.426114 (+   390us) log.cc:844] Serialized 77 byte log entry
0709 12:18:07.427155 (+  1041us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:07.427372 (+   217us) write_op.cc:454] Released schema lock
0709 12:18:07.427445 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:07.427973 (+   528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:07.428020 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":355902,"apply.run_cpu_time_us":4835,"apply.run_wall_time_us":207059,"num_ops":10,"prepare.queue_time_us":460069,"prepare.run_cpu_time_us":17513,"prepare.run_wall_time_us":281408,"raft.queue_time_us":379089,"raft.run_cpu_time_us":20712,"raft.run_wall_time_us":222858,"replication_time_us":807717,"thread_start_us":834809,"threads_started":4,"wal-append.queue_time_us":60819,"wal-append.run_cpu_time_us":70710,"wal-append.run_wall_time_us":2365790}]]}
I20260709 12:18:08.887075 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:08.889261 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.889492 18293 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:18:08.890861 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.896257 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.900987 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.902257 19042 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:08.862198 22173 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:18:08.904510 18374 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:08.912325 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.899226 21947 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.911185 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.914921 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.916797 18194 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:08.903807 22169 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.906328 22104 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:08.930003 22104 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.930908 21818 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:08.931661 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:08.931488 21818 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.932301 22104 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.932897 22104 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.933250 22104 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:08.933703 22104 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.920904 22190 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.906571 22149 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.935400 22149 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:08.935175 22190 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.935585 18293 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:08.935810 21648 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:08.936653 19044 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:08.936548 21729 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.937392 22190 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.937660 19039 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:08.938755 21729 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.924126 21926 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:08.939904 21926 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.940276 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:08.941502 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.941884 21926 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.927326 21923 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.923502 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.942960 18374 raft_consensus.cc:2410] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:08.927578 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.929384 22185 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.923663 21577 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.928489 21722 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.942602 21923 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.936348 21648 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.942049 19038 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
W20260709 12:18:08.944872 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.946224 18372 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:08.954037 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.955641 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.956512 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.960644 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.962818 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:08.935832 22149 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.964299 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:08.964926 18292 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:08.967068 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.933714 21818 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.968075 18372 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:08.969899 22148 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:08.962625 18294 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.970556 22148 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.972282 19039 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:08.972692 22148 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:08.972273 18294 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:08.973429 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.961750 18194 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:08.981981 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.983371 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:08.984031 18442 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:08.918532 21947 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:08.987718 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:08.988409 18375 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:08.989387 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:08.989987 18371 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:08.952139 22190 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:08.994104 22091 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.959520 21577 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:08.996331 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:08.996889 18442 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:08.997233 21577 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:08.998716 21923 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.000828 19041 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:18:09.003409 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.004730 19040 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:09.015518 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:09.019564 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 61238.3us
I20260709 12:18:09.011736 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.024273 18371 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
W20260709 12:18:09.026659 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44355.7us
I20260709 12:18:09.030483 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:09.023027 18366 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:09.028802 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:09.047317 18369 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:09.051833 18372 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:18:09.051945 22078 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.052915 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.059201 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.034682 21648 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.065968 18194 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.063488 18442 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:09.073277 18374 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:09.075655 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.067639 21923 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.075791 18194 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.076880 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.084091 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.084995 18448 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:09.085094 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:09.087179 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 679) took 1004 ms. Trace:
I20260709 12:18:09.084106 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:09.087832 19042 rpcz_store.cc:276] 0709 12:18:08.082928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.084293 (+  1365us) service_pool.cc:224] Handling call
0709 12:18:09.086967 (+1002674us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:09.087096 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504070016}
I20260709 12:18:09.088372 19036 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:09.087131 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.089782 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.090583 18440 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:09.084841 21631 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.091598 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.096199 18443 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.096495 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.100943 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:09.102807 18375 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:09.103093 18298 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:09.101054 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:09.104342 18442 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:09.105060 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.105443 18442 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:09.079429 21923 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.106947 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.107620 18374 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:09.107343 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.109530 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.110181 18447 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:09.111025 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.111471 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.107548 18445 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:09.100342 18194 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.117817 21930 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.118613 21930 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.121392 19039 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:09.106956 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.123790 18441 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:09.127563 18298 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:09.127166 22137 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:09.130342 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.102993 22195 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.102665 22091 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.130533 18194 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 6 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 7 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:09.133404 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.137655 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.112969 21923 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.149385 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.144570 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:09.149019 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.144977 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:09.148651 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.166574 18442 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 6.
I20260709 12:18:09.167397 18298 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.172134 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.173038 19038 raft_consensus.cc:2368] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10.
I20260709 12:18:09.144498 22216 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:18:09.170903 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.183809 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.184757 18296 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:09.188470 18374 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:09.188021 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.193589 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.194725 18292 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:09.137228 22195 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.190410 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:09.145887 21947 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:09.200201 21930 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.143122 21722 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:18:09.211633 21722 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:09.212316 21722 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:09.212723 21722 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.213714 21722 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.214105 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.214804 18373 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:09.215796 18366 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:09.216835 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.219331 18371 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:09.221463 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.222134 18368 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:09.219357 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 74567.6us
I20260709 12:18:09.211655 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.200812 22107 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:09.223656 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.230574 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.203024 19039 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:09.190320 18194 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.204634 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.200733 19042 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:09.228384 21631 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.231249 18296 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:09.239399 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.231768 19042 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.248370 18263 leader_election.cc:400] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10.
I20260709 12:18:09.248857 18263 leader_election.cc:403] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:09.243685 18448 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:09.326979 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.277331 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.341874 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:09.288235 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64659.6us
I20260709 12:18:09.346192 18375 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:09.346405 18292 raft_consensus.cc:2410] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:09.341638 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.326092 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.308736 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.269366 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.300930 18369 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:09.215034 21722 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.288851 18194 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.352793 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.353232 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.353720 18297 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.353951 18194 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 2 to 5, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 5 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:09.346700 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.350368 18298 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:09.351629 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.352340 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.352770 18296 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:09.360913 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.362809 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.359879 18367 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:09.355130 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.364820 18445 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:09.365247 18369 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:09.360248 18194 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 7 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.369618 18297 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:09.324545 19042 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.230911 22091 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.372058 21930 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:09.378793 21930 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.379160 21930 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.373940 21648 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:18:09.380280 21648 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:09.380668 21648 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.381692 21648 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.382958 21648 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.303896 21631 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:09.383953 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.385294 21818 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:09.408524 18367 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:09.387498 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.388607 21729 raft_consensus.cc:993] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:18:09.388454 22117 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:09.389108 21947 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10.
I20260709 12:18:09.389508 18446 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.422130 19042 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.389894 21983 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:18:09.424271 21983 tablet_bootstrap.cc:492] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:18:09.425045 21983 ts_tablet_manager.cc:1403] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 1.361s	user 0.051s	sys 0.030s
I20260709 12:18:09.389976 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:09.390720 22172 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:09.393113 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.312130 22195 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.315480 22149 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.393970 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.379511 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.436303 18370 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.399384 18194 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.437623 18194 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:09.403904 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.405503 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.387108 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:09.409052 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 203072us
I20260709 12:18:09.410710 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.409219 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.408507 21631 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:09.412124 21818 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:09.412714 21729 raft_consensus.cc:1081] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Signalling peer e23c0ce8d89940da8eac6a6002e349c5 to start an election
I20260709 12:18:09.410066 22091 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.310469 21923 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.439904 22169 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.413735 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:09.410466 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.416466 19043 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:09.426460 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.445024 18300 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:09.446161 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:09.486538 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60170.5us
I20260709 12:18:09.486892 18297 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:09.487192 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.452190 18375 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:09.455344 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:09.457108 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:09.457386 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.458344 22207 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.458788 21921 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.459230 22190 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:09.460312 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.461561 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.462050 21631 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:09.471676 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:09.504814 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49578.1us
I20260709 12:18:09.505652 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:09.474812 22149 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:09.507107 21577 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:09.508383 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51371.5us
W20260709 12:18:09.508847 22223 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.328s	user 0.005s	sys 0.000s
I20260709 12:18:09.509164 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.510883 18266 leader_election.cc:400] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10.
I20260709 12:18:09.511404 21856 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.511788 21856 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.512118 21856 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.514410 19042 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:09.515349 22207 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.517594 22207 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:09.518106 22207 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:09.427354 21968 consensus_queue.cc:1175] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e23c0ce8d89940da8eac6a6002e349c5" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 12:18:09.475119 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.418921 22104 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.413336 21930 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.443918 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:09.485882 22169 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:09.438997 18194 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 7 to 10, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.537344 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.496775 18446 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:09.479564 22033 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:09.475544 21818 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.500603 18291 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:09.501494 18369 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:09.494098 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 107053us
I20260709 12:18:09.539597 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.495083 22216 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:09.528276 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.531937 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:09.535653 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 91799.3us
I20260709 12:18:09.484267 18445 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:09.494642 22172 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:09.542600 18370 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:09.544359 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.546847 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:09.547958 22050 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:09.549851 21983 raft_consensus.cc:359] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.551291 21818 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:09.552386 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 45276us
I20260709 12:18:09.556010 22172 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.556388 22216 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:09.557982 22091 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.558414 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.540660 22169 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.540822 21947 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:09.560730 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.562608 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.521929 22190 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:09.573662 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.574388 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:09.575827 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.577795 21930 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:09.521770 21921 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.579955 22104 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:09.580510 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.580873 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.538434 21945 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:09.584054 22230 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:18:09.586645 21923 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:09.540627 18447 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
W20260709 12:18:09.588254 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47975us
I20260709 12:18:09.588765 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.589294 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.590494 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.596567 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.605520 18373 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:09.607854 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.604542 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:09.614984 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:09.617933 18443 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.006s
I20260709 12:18:09.626907 18368 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:09.627555 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:09.671144 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:09.678782 18443 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
W20260709 12:18:09.678941 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51473.2us
I20260709 12:18:09.679600 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.679829 18442 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:09.669031 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.680831 18446 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.675290 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:09.690945 18370 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:09.655925 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.640633 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:09.681066 18375 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:09.605592 22216 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:09.702746 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:09.703474 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:09.674885 22229 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.131s	user 0.002s	sys 0.003s
I20260709 12:18:09.704072 18372 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:09.708752 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.708923 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.709303 18372 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:09.703385 18448 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:09.705744 18447 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:18:09.704944 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104961us
I20260709 12:18:09.710388 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.711088 18372 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:09.627650 21947 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.673301 22107 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:09.639954 21916 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:09.713898 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:09.707827 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 230170us
I20260709 12:18:09.715963 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.719177 18373 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:09.716027 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.695703 22050 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:09.694655 22148 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:09.716229 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 169430us
I20260709 12:18:09.723014 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.716933 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.718516 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:09.723917 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:09.703001 22199 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:18:09.724084 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:09.734496 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:09.724943 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:09.723989 18368 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.669482 21635 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:09.738854 18372 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:09.733301 18446 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:09.698164 22149 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:09.734094 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.609381 21856 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.765030 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:09.765906 18372 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.701680 21577 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:09.754880 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.718257 21947 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.762153 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.761323 18369 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:09.769284 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.771478 18371 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:09.773355 18375 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:09.718822 22107 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.722843 22050 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:09.738552 22104 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.789922 18373 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:09.753811 22149 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.799045 18449 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:09.767369 21945 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:09.766683 22234 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.807121 18444 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:18:09.813877 18444 pending_rounds.cc:85] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 3
I20260709 12:18:09.814270 18444 pending_rounds.cc:107] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:18:09.814599 18444 raft_consensus.cc:2889] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:09.775404 22137 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:09.787595 22033 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:09.829655 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 120792us
I20260709 12:18:09.773494 21983 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:18:09.739530 22148 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.832715 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.796370 22149 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:09.773216 21916 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 12:18:09.836444 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68760.2us
I20260709 12:18:09.806109 21945 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.761090 21635 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.811650 21648 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:09.834308 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 65096.9us
I20260709 12:18:09.796211 22104 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.878262 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:09.880528 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118490us
I20260709 12:18:09.881279 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:09.877745 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 89386.2us
I20260709 12:18:09.886623 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:09.838620 22033 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:09.870779 22137 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.834267 22239 raft_consensus.cc:2764] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:09.775039 22172 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.892508 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:09.872751 21945 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.876881 21648 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:09.906486 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 231343us
I20260709 12:18:09.908680 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:09.913456 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80827.7us
I20260709 12:18:09.915266 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:09.882962 22235 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.888121 21722 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.841924 21930 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:18:09.895324 22234 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:09.889382 22245 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:09.923969 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:09.912704 21648 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:09.934808 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53603.7us
I20260709 12:18:09.916545 22078 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:09.853220 22107 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.930037 22091 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.867996 22050 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:09.921424 22242 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.930158 21921 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.883124 21983 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:09.953442 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61005.1us
I20260709 12:18:09.938875 22078 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.934561 22247 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:09.957265 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.901672 21577 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.943993 22091 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:09.962752 22232 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:18:09.962951 22237 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:18:10.009227 18372 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:09.962364 22033 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.963276 22246 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:18:09.963773 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:09.964329 22240 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:09.958112 21856 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:09.969350 22107 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.975999 21631 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.980672 21635 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:09.955751 22245 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:09.928166 22235 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:09.985189 22248 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:09.952135 21916 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:09.989151 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:10.009230 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51997.1us
I20260709 12:18:10.011312 18368 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:09.959502 22078 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.953146 22251 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.021836 22251 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:10.022048 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.991377 22091 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.023775 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:09.974776 22169 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.024616 22239 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.025249 22239 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.013437 22236 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:10.022228 22251 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:10.019192 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 105193us
I20260709 12:18:10.016899 22249 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:10.028986 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65359us
I20260709 12:18:10.030691 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:10.030737 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.017580 21722 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:10.020745 22248 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.034446 22248 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.035197 21983 ts_tablet_manager.cc:1434] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.610s	user 0.008s	sys 0.004s
W20260709 12:18:10.035585 21983 ts_tablet_manager.cc:1467] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Tablet startup took 610ms
I20260709 12:18:10.033206 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.040792 22238 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:10.041388 22238 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:10.042357 22231 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.559s	user 0.003s	sys 0.002s
I20260709 12:18:10.042325 22238 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.044282 21631 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:10.048372 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 91405.9us
I20260709 12:18:10.049428 22104 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:10.051604 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.053074 22078 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.030400 22249 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:10.106696 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 72763.4us
I20260709 12:18:10.109409 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.110517 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.032574 21968 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.111646 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:10.112936 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:10.113909 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:10.115267 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84546us
I20260709 12:18:10.118728 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:10.117714 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:10.120760 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.121098 19035 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:10.117262 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:10.122035 19041 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:10.124713 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 93722.1us
I20260709 12:18:10.029258 21916 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.132331 22247 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:10.132750 22247 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:10.132897 21916 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.114657 22249 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:10.131115 19038 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:10.128250 22245 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.130271 22149 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.108830 21856 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:10.149390 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.055207 22078 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.030161 21635 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:18:10.074436 21631 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.027514 22252 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:18:10.150861 21631 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.159123 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.162562 18366 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.166720 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.124260 22104 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.119686 22169 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.105731 21648 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.154762 21916 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:10.219241 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 112794us
I20260709 12:18:10.165602 21577 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:10.175192 22137 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:10.239234 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 115324us
I20260709 12:18:10.223919 21916 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.240443 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:10.224105 21856 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.244185 22248 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.246379 22248 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.247645 21945 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.230932 22104 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:18:10.249004 21631 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.226052 22190 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:10.252607 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:10.255649 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.272850 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:10.270284 22256 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:10.252604 21577 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.273488 18445 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:10.219947 21947 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.250989 21916 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.249794 21856 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:10.035897 21983 ts_tablet_manager.cc:1469] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Trace:
0709 12:18:01.586459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.593901 (+  7442us) service_pool.cc:224] Handling call
0709 12:18:02.383387 (+789486us) ts_tablet_manager.cc:934] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
0709 12:18:04.484568 (+2101181us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:05.650511 (+1165943us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:05.650628 (+   117us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:06.547458 (+896830us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:06.631753 (+ 84295us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:06.775544 (+143791us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:06.781208 (+  5664us) inbound_call.cc:177] Queueing success response
0709 12:18:08.063856 (+1282648us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:18:09.425385 (+1361529us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:18:09.453233 (+ 27848us) tablet_replica.cc:261] Starting instrumentation
0709 12:18:09.453899 (+   666us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":40330,"mutex_wait_us":26116,"raft.queue_time_us":25693,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":88,"tablet-copy.queue_time_us":380584,"tablet-download-pool.queue_time_us":126,"tablet-download-pool.run_cpu_time_us":1406,"tablet-download-pool.run_wall_time_us":281860,"thread_start_us":323857,"threads_started":2}
I20260709 12:18:10.279822 22247 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.280859 22247 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.283118 18443 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:10.285517 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.251212 22235 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.292172 22262 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:18:10.292944 22262 consensus_queue.cc:1243] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:10.293500 21648 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.293540 21818 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.294786 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.295506 18443 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.300109 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-67a1783f49244bd5ba98721205d1ff6e received from {username='slave'} at 127.0.0.1:46956
I20260709 12:18:10.300654 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-67a1783f49244bd5ba98721205d1ff6e on tablet 67a1783f49244bd5ba98721205d1ff6e with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.301167 21577 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.302179 19041 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:10.307771 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.308750 22234 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:10.308897 22148 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:10.309252 22234 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:10.293391 22262 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:10.316691 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:10.283382 21856 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.274729 22248 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:10.329981 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 111178us
I20260709 12:18:10.284065 22256 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.332505 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.333030 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:10.331633 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47548.9us
I20260709 12:18:10.345036 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:10.345386 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60027us
I20260709 12:18:10.346544 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.346546 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:10.275303 22252 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.054s
W20260709 12:18:10.349690 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 239218us
I20260709 12:18:10.237607 21722 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.353447 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:10.326203 22262 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:10.353524 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:10.359593 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 308056us
I20260709 12:18:10.331991 22248 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.330569 22148 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.363832 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.307122 22251 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.364450 18443 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.366803 18447 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:10.369622 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.370256 18445 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.352291 22235 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:10.372579 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:10.373279 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.354676 21648 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:10.352172 22107 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.373796 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:10.360975 22033 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.372902 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.384773 18447 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:10.364327 22248 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:10.387720 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55299.4us
I20260709 12:18:10.389346 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.387964 18445 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:10.391561 19039 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:10.372294 22245 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.358412 21923 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:18:10.374953 22235 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.384114 22149 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.399694 18443 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:10.386052 22107 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:10.401554 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 282914us
I20260709 12:18:10.387706 22033 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.402650 22270 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.386998 22148 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:10.404291 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:10.406286 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52959.6us
W20260709 12:18:10.406492 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61345.1us
I20260709 12:18:10.407451 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.392628 21648 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:10.409682 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:10.415134 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:10.437891 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:10.401299 22216 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.662s	user 0.006s	sys 0.005s
I20260709 12:18:10.403970 22107 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.406215 22033 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.401173 22235 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.479875 21947 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:10.487655 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49879.2us
I20260709 12:18:10.494277 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.489001 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:10.493264 18366 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:10.492857 21648 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.473218 22091 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.502516 22078 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:10.463708 22236 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:10.505743 22033 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.508021 22251 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.508692 19043 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:18:10.509219 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 106922us
I20260709 12:18:10.510186 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:10.366063 22234 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.475217 22169 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:10.493460 22270 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:10.538329 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 134123us
I20260709 12:18:10.552634 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:10.556496 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:10.557154 18370 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:10.558725 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.396454 22262 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.513010 21945 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.551782 22172 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.579911 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:10.563714 22269 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:18:10.564842 22267 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 12:18:10.565290 22241 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:10.568531 22235 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.568670 22248 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.572995 21856 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.505743 21916 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.575305 21577 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.403512 22256 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:10.594825 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 221989us
I20260709 12:18:10.488029 22216 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.595665 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:10.615442 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 127239us
I20260709 12:18:10.580068 21818 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:10.539978 22149 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:18:10.587560 22251 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.578410 22169 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:10.665233 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 69641.8us
I20260709 12:18:10.674659 19042 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:10.674999 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:10.594425 22270 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.627887 22172 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.665050 22273 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.595947 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.679018 22270 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:10.679256 22265 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.679654 22279 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.708250 22033 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.708365 22279 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.709012 22279 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.698989 22172 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.709522 22279 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.673909 21818 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:10.708202 21916 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.708173 21856 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.587263 22253 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:18:10.683574 22251 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:18:10.684226 22249 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.001s
I20260709 12:18:10.666076 22262 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.684765 22148 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.685725 22216 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:10.629004 22245 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:10.702068 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 106208us
I20260709 12:18:10.712021 21818 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.677802 21648 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.687588 22235 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.699358 22273 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.713825 21947 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:10.703320 22270 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.686547 22169 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.716835 22253 consensus_queue.cc:1243] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:18:10.722450 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 210957us
W20260709 12:18:10.771590 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 382290us
I20260709 12:18:10.773221 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.722148 22172 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:10.785858 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:10.787094 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:10.777019 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 282820us
I20260709 12:18:10.787725 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.788249 18445 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:18:10.779106 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 104199us
I20260709 12:18:10.789151 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:10.790370 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46726us
I20260709 12:18:10.791736 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:10.741317 22148 raft_consensus.cc:2764] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:10.725390 22273 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.794471 18297 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:10.794327 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:10.717681 22249 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.797710 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.798007 18445 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:10.792250 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.774950 22216 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.798534 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.800812 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.801523 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.803329 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.799103 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.791736 21947 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.805297 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.805945 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.776808 22033 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.788839 21923 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.817276 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.818889 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.799660 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.799077 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.803063 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.804543 18295 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:10.725363 21648 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.809036 18443 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.778857 22282 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:10.810475 18299 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:10.789809 22248 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.723482 22245 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.827589 22286 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.827883 22245 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.828059 22286 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:10.828531 22286 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.830086 22280 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:10.831530 22283 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.822518 22282 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.834885 22282 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.836119 22282 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:10.836501 22288 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:10.837000 22288 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.837774 22288 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.818329 22033 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.842900 22149 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:10.843322 22249 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.843444 22149 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.843904 22283 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:10.844784 21856 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:10.845137 21856 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.845602 22149 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:10.845865 21856 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.846836 21856 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:10.849139 21648 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:10.849996 21947 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.818351 22238 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.790050 21635 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.853202 21635 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:10.853605 21635 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.809465 22216 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.801003 22265 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.857402 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:10.802667 21577 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.878649 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:10.879280 18296 raft_consensus.cc:2376] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
I20260709 12:18:10.880093 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.777581 22252 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:18:10.880908 22091 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.806362 22251 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.881712 22251 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.882117 22251 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.883526 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.818130 21923 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.885848 21923 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.842661 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:10.886803 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44212.9us
I20260709 12:18:10.889223 22033 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:10.889672 21968 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.890182 21968 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.892663 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.893232 18448 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.894148 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:10.894686 19042 raft_consensus.cc:606] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:10.895275 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.895478 18292 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:10.895933 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.896597 22242 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:10.896790 18440 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:10.897100 22242 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:10.897984 22242 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:10.776352 22107 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.806507 21921 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:10.877048 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 59878.3us
I20260709 12:18:10.804149 22137 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 12:18:10.899878 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 107705us
I20260709 12:18:10.901410 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.922418 18366 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.902274 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.903586 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:10.927814 18371 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:10.929098 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.930037 18371 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:10.930627 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.931852 18369 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:10.904071 18443 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:10.823870 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.932754 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:10.910835 18292 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:10.934373 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:10.920233 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:10.932858 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.935189 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.935601 18372 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.917402 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.937497 18367 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.937898 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:10.938285 18445 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:10.939656 18440 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:10.940389 18368 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:10.882843 21635 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.941885 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.946447 18373 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.947067 18444 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:10.947127 18366 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:10.948148 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:10.943053 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:10.949563 19043 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.899742 22050 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.942430 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:10.903482 22265 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.944571 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.951669 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:10.954653 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:10.955536 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 131732us
I20260709 12:18:10.913054 21945 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:10.956458 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:10.922209 21921 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:11.154548 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:10.953485 18372 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:11.025995 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:11.154687 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 206612us
I20260709 12:18:11.160616 18371 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: [9.10-11.11]
I20260709 12:18:11.160825 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:10.993700 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:11.107470 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 64911.7us
I20260709 12:18:11.032415 22107 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:11.031535 22287 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.199s
I20260709 12:18:11.158010 18370 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:10.958562 18296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:43604
I20260709 12:18:10.997429 18293 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.032279 22284 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.203s
I20260709 12:18:10.996495 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:11.000169 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:10.992174 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.160044 18291 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:18:11.161895 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.044843 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.177665 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.163563 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.023279 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:11.170539 18445 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:11.179051 19041 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:11.170552 18448 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:10.959690 18374 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:11.179313 18296 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.180275 18296 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:11.180354 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 135618us
W20260709 12:18:11.181272 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 224903us
I20260709 12:18:10.958264 22091 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.949277 22207 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:10.958334 22296 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:18:11.185001 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.185619 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:11.185909 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.178468 18295 raft_consensus.cc:2376] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10. Re-sending same reply.
I20260709 12:18:11.197600 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.191704 18367 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:10.996793 22104 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:11.036280 21635 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.145169 21923 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:11.161095 21968 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:11.207749 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 295473us
I20260709 12:18:11.161034 22216 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:11.205175 18297 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
W20260709 12:18:11.210736 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47227.8us
I20260709 12:18:10.955229 22265 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.244681 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.192639 21945 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:11.028941 22270 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:10.954166 22148 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.248s
I20260709 12:18:11.240381 18370 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:11.240716 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.032704 22273 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:11.191042 22238 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:11.200838 22281 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:11.191576 22091 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:11.247987 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:10.954784 22172 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.280s
I20260709 12:18:11.205367 22253 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:11.201148 22285 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.201551 22195 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.206606 22248 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.206943 22262 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:11.295379 22262 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:11.295864 22262 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.241571 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.254601 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:11.300114 18372 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:11.031723 22293 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:11.209860 22104 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:11.300679 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.301395 22104 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.301463 18295 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.193832 22288 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.192028 22286 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.276692 18367 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.191123 22279 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:11.310397 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.310302 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:11.202531 22207 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.253644 18300 raft_consensus.cc:2376] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10. Re-sending same reply.
I20260709 12:18:11.251613 22271 raft_consensus.cc:2764] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:11.324162 18374 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:18:11.250689 21968 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:11.327610 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 124267us
W20260709 12:18:11.328146 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 130501us
I20260709 12:18:11.337718 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.278512 22091 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.242261 21923 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.340368 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:11.346864 18367 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:11.356361 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.335991 22148 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:11.345415 22256 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:11.357273 22148 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:11.357767 22256 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:11.358062 22270 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.358192 22256 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.359047 22270 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:11.358289 22148 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.306726 22285 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.359581 22270 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:11.360144 22148 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.360535 22207 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.361200 22285 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:11.361662 22301 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:11.362390 22279 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:11.363598 18447 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:11.358075 19041 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:18:11.369408 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.369830 22273 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:11.371142 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.375423 19042 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:11.376559 18367 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:18:11.277974 22235 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:18:11.377772 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.279222 21945 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:18:11.397749 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.401841 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.398259 18366 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:11.252857 22251 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.382855 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.413815 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.408818 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.416697 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.417440 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:11.419009 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.418206 18367 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
I20260709 12:18:11.249204 21921 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.421849 18366 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:11.422715 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.467826 18375 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:11.468088 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:11.472445 19043 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.004s
I20260709 12:18:11.472726 18295 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:11.378129 22238 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:11.475010 18294 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:11.477807 18373 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.003s
I20260709 12:18:11.478648 18373 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:11.446242 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:11.447790 22137 consensus_queue.cc:1175] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" responder_term: 7 status { last_received { term: 7 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 7 index: 6 } }
I20260709 12:18:11.360551 22270 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:11.449024 22299 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:18:11.382288 22284 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.452539 22273 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:11.417322 22148 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.458818 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.459419 22288 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:11.461810 22262 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.465602 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.501677 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.420392 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:11.502506 18367 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.473169 19043 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:11.479719 19042 raft_consensus.cc:606] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:11.471624 18296 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.481112 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:11.475385 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146523us
I20260709 12:18:11.507975 18300 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:11.508948 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.509527 22262 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:11.484951 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.443893 21921 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.474704 22238 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.492594 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.515609 22238 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.516371 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.517045 18366 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:18:11.495611 21577 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.518201 21577 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.520174 21577 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.521271 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.496027 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:11.522873 22238 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.533605 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:11.465612 22286 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:11.509737 18300 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:11.483345 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.514653 18367 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:11.522693 18295 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:11.500000 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:11.538722 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55519.8us
I20260709 12:18:11.498529 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.539997 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:11.522914 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.555783 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:11.563097 18366 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:11.563953 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:11.540409 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.568866 18375 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:11.570254 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 101143us
I20260709 12:18:11.571008 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:11.501117 22149 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.572206 18366 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:11.572978 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:11.574265 22265 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.575090 22091 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.562304 21921 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:11.571895 22312 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:11.500641 22256 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.656652 21921 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:11.567471 22253 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:18:11.657721 22253 consensus_queue.cc:1243] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:11.657989 22091 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.657635 21921 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.658366 22256 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.580580 22320 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:11.659183 22251 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:11.659340 18298 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:11.581385 21945 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:11.658393 22253 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:11.660211 21945 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:11.660427 22253 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:11.661168 21945 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.661326 22253 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.588426 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.662417 18300 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.580593 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.588851 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:11.639160 19042 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:11.562616 22262 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:11.639870 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50024.6us
I20260709 12:18:11.540773 22248 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.653745 18295 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:11.661849 22238 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:11.640594 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1528832649999003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:18:11.566862 22190 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:11.638935 22268 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.667193 18299 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:11.898183 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.675369 22312 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:11.823040 22265 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:11.834000 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 294630us
I20260709 12:18:11.667081 21577 raft_consensus.cc:2764] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:18:11.854848 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.673192 22324 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:11.877091 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 288305us
I20260709 12:18:11.900426 18375 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:11.681257 21923 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.900830 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:11.899910 22253 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.679191 21968 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:11.902874 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:11.672587 22317 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.231s
I20260709 12:18:11.903606 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:11.904477 18442 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:11.904726 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:11.905885 19043 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:11.905967 22033 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.676976 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.906628 22328 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:11.663070 22313 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:11.806521 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.911317 22190 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:11.900256 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 319707us
I20260709 12:18:11.907644 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2280) took 1124 ms. Trace:
I20260709 12:18:11.911998 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:11.926949 18292 rpcz_store.cc:276] 0709 12:18:10.784456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.792025 (+  7569us) service_pool.cc:224] Handling call
0709 12:18:11.907463 (+1115438us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b540004d154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:11.907582 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225653504}
I20260709 12:18:11.929365 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:11.930159 21577 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:11.914011 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:11.916755 22326 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:18:11.916646 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:11.931577 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:11.918354 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:11.775309 22288 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.920531 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:11.910353 22234 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:11.920231 19042 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:11.921630 18440 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:11.921664 18444 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:18:11.933313 18444 pending_rounds.cc:85] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 4
I20260709 12:18:11.907202 22329 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.911432 22324 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.924885 18375 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:11.925618 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.676556 18372 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.930636 21577 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:11.931874 18443 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:11.917351 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 792) took 1015 ms. Trace:
I20260709 12:18:11.937747 21577 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.782183 22104 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:11.933548 18444 pending_rounds.cc:107] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:11.933456 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.942118 18444 raft_consensus.cc:2889] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:11.942524 18300 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:18:11.940176 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:11.936753 18375 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:11.938004 18370 rpcz_store.cc:276] 0709 12:18:10.901565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.902021 (+   456us) service_pool.cc:224] Handling call
0709 12:18:11.917189 (+1015168us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54000f9454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:11.917290 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529810688}
I20260709 12:18:11.911844 22268 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.943607 18446 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:11.946537 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.941859 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:11.935650 22149 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.947480 18292 raft_consensus.cc:2410] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:11.934144 22091 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:18:11.947723 22149 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.948275 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.949026 18292 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:11.949779 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.949884 22149 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.940914 22248 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.950537 22248 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.950884 22268 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.946420 22331 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:11.951949 22331 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:11.950814 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.952693 22248 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:11.940277 22316 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:11.925272 22033 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.952870 22331 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.945210 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.955080 22033 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:11.953264 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.955672 18300 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:11.955654 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:11.944969 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:11.955680 18292 raft_consensus.cc:2376] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9. Re-sending same reply.
I20260709 12:18:11.924300 21968 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:11.965174 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:11.966972 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.955590 22033 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.968808 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.968981 22033 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.969915 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:11.958657 18375 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.971915 18375 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:11.944195 21923 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:11.977082 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62183us
I20260709 12:18:11.927398 22265 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:18:11.981282 18441 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.982228 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:11.942063 22324 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.983042 19042 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:11.986433 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:11.970710 22256 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:11.971694 22270 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.990635 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:11.977398 22149 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:11.927119 22328 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.941669 22288 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.990818 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:11.990669 19042 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:11.993319 19041 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:11.991780 19044 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:11.997666 22337 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:11.998149 22337 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.998674 22337 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.007292 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.007948 19038 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:12.007694 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:11.928574 22313 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.010753 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.015167 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.015672 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.016572 18445 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:11.942775 22329 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.023102 22329 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.024667 22328 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:11.956815 22316 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.026168 22288 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.027545 22256 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.028018 22256 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.028481 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.029585 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.029968 22256 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.030283 18300 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:11.984494 22190 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.026968 22316 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.031008 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.034109 22336 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:18:12.019433 21968 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:11.982025 21923 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:12.040735 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 72864.9us
I20260709 12:18:11.994210 22238 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:18:12.042384 18291 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
W20260709 12:18:12.036509 22332 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:11.995290 22270 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.045498 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.048290 18292 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.048763 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.049364 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:12.052659 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.053234 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.053970 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.054049 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:12.055959 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.056524 18291 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:12.057044 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.057420 18291 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.064801 21577 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:11.983171 22312 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.071393 19035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:12.072016 19035 raft_consensus.cc:606] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.060642 21945 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:18:12.072378 19035 raft_consensus.cc:614] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:12.061069 22328 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:12.073714 22328 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:12.076625 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:12.078671 21577 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.060050 22316 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:12.084239 18291 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:12.074520 22328 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.087373 22254 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.088080 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.080315 21945 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:12.091321 21945 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.091677 21945 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:12.095568 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.096199 18446 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:12.098115 22270 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:12.142781 18375 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:18:12.143292 18375 pending_rounds.cc:85] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:12.143553 18375 pending_rounds.cc:107] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:12.143818 18375 raft_consensus.cc:2889] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:12.145486 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.083465 22337 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.231719 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:12.232355 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.524s (SENT)
I20260709 12:18:12.232725 18446 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:12.156778 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 1982) took 1370 ms. Trace:
I20260709 12:18:12.233327 18447 rpcz_store.cc:276] 0709 12:18:10.786544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.793233 (+  6689us) service_pool.cc:224] Handling call
0709 12:18:12.156481 (+1363248us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54002aa154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:12.156656 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248169856}
I20260709 12:18:12.234859 18447 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.236390 18447 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:12.242110 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:12.243199 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97413.7us
W20260709 12:18:12.177965 22327 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
W20260709 12:18:12.185042 18299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1852) took 1352 ms (client timeout 1615 ms). Trace:
I20260709 12:18:12.187153 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1963) took 1279 ms. Trace:
W20260709 12:18:12.188771 22333 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 12:18:12.192438 22302 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.889s	user 0.003s	sys 0.001s
I20260709 12:18:12.193120 18372 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:12.194932 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.215209 22033 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:12.219450 22300 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.926s	user 0.000s	sys 0.004s
W20260709 12:18:12.223119 22297 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.929s	user 0.000s	sys 0.004s
I20260709 12:18:12.229704 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:12.280978 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48671.2us
I20260709 12:18:12.282482 18366 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:18:12.272549 18299 rpcz_store.cc:269] 0709 12:18:10.832657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.856957 (+ 24300us) service_pool.cc:224] Handling call
0709 12:18:12.184771 (+1327814us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400330254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:12.184938 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763528960}
I20260709 12:18:12.273362 18371 rpcz_store.cc:276] 0709 12:18:10.908163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.946676 (+ 38513us) service_pool.cc:224] Handling call
0709 12:18:10.946954 (+   278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:11.161074 (+214120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.186820 (+1025746us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400395154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:12.187081 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490714368}
I20260709 12:18:12.283393 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.284219 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.298730 18375 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:12.286027 19043 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.282995 18369 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:12.299337 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.299674 18442 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.299965 18291 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:12.300818 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.301421 18291 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:12.288861 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.302743 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.303050 22287 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:12.303874 22287 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:18:12.304288 22343 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:18:12.304358 22287 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:12.304772 22287 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:12.305078 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.305548 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.306492 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.306670 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.307083 18444 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:12.308086 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.309015 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.310760 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.310770 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.312611 18299 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:12.313206 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.314035 19035 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:12.287575 18292 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:12.287223 21923 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.293095 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.322351 18446 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:12.321638 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.322981 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.323597 18371 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.324002 18375 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:18:12.324276 19041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:12.324242 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.325057 19041 raft_consensus.cc:606] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:12.325103 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.322911 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:12.325560 19041 raft_consensus.cc:614] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:12.326747 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.090230 22254 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.331907 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.332751 18291 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:12.333878 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.334589 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:12.328945 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.338604 18375 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:12.338697 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.329078 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.328522 18366 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:12.341300 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.343633 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.344621 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.345353 18440 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:12.349128 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.283704 22169 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:18:12.349807 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.349773 18442 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:12.351719 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:12.361409 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.363324 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.365914 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.374817 18291 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:12.375497 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 700) took 1399 ms. Trace:
I20260709 12:18:12.376415 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.377635 18371 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:12.377930 18299 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.378614 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 701) took 1402 ms. Trace:
I20260709 12:18:12.379014 18299 rpcz_store.cc:276] 0709 12:18:10.976310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.347251 (+1370941us) service_pool.cc:224] Handling call
0709 12:18:12.377769 (+ 30518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.378353 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.378562 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.337160 21729 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.337574 22148 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.393929 22148 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:12.378266 22293 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.395079 22148 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:12.364791 22107 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 12:18:12.399343 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55773.5us
I20260709 12:18:12.332782 22313 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.358614 21923 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:18:12.400768 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.400671 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.401571 18440 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:12.350952 22253 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:18:12.351280 21968 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:12.402432 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:12.402779 21968 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.402899 22284 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.403264 21968 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:12.404112 22137 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.404520 22137 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.404837 22137 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:12.407392 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.375802 18291 rpcz_store.cc:276] 0709 12:18:10.976001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.348319 (+1372318us) service_pool.cc:224] Handling call
0709 12:18:12.374610 (+ 26291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.375256 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.375445 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.409076 18187 maintenance_manager.cc:419] P 79114be45f9144e78102be8765ae6af5: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033381
I20260709 12:18:12.342760 21947 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.411458 21947 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.377681 22331 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.413333 22331 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.413825 22331 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.402874 18412 leader_election.cc:403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:12.416337 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.383817 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.337613 22252 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.385315 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.359015 22234 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.387097 18299 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:12.349354 22341 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.390340 18371 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.391403 21729 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:12.392979 22337 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:12.387880 21722 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.421944 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 706) took 1444 ms. Trace:
I20260709 12:18:12.430631 18299 rpcz_store.cc:276] 0709 12:18:10.977696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.386390 (+1408694us) service_pool.cc:224] Handling call
0709 12:18:12.386740 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.420255 (+ 33515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.421874 (+  1619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.424177 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.414934 21947 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.426386 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:12.419970 22313 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.305564 22287 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.434527 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.424362 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.423024 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:12.421994 22341 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.421530 22234 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:12.468223 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.433730 22253 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.430428 21722 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:12.478251 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53987.6us
I20260709 12:18:12.491856 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:12.481369 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.482230 18298 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:18:12.495282 18298 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:12.482486 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:12.473165 22311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.011s	sys 0.019s
I20260709 12:18:12.488541 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.419829 22265 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:12.454607 22247 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:12.454741 21947 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.448894 22252 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.470861 22234 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:12.497980 22324 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
I20260709 12:18:12.454587 22285 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.518987 18298 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:18:12.470232 22341 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.522284 19039 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:12.493561 21722 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.490773 22253 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.493212 22245 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:12.496565 22287 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:12.521103 22262 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.486359 21635 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:12.475612 22195 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:18:12.492468 22169 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 12:18:12.536813 22311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.011s	sys 0.026s
I20260709 12:18:12.520502 22337 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:12.522274 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.520910 22270 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.523718 21947 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.520635 22239 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.493925 22260 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:18:12.581561 19043 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.572173 22253 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:12.591970 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.586338 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.572021 22256 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.610811 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.546984 22247 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.563850 22324 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.579461 21818 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:12.569383 22252 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.591502 21947 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.567019 22285 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.486925 21729 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.589488 22337 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.599136 22338 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:12.627694 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:12.632696 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110485us
I20260709 12:18:12.633524 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.580675 22245 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.580284 21945 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.622531 22340 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.648486 18293 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:18:12.648862 18293 pending_rounds.cc:85] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 8
I20260709 12:18:12.649087 18293 pending_rounds.cc:107] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:18:12.649307 18293 raft_consensus.cc:2889] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:12.649791 18415 leader_election.cc:336] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 2.005s (SENT)
I20260709 12:18:12.650223 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:12.651364 22340 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.627894 22078 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.651827 22340 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.596417 22239 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.653384 18373 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:18:12.628309 22337 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:12.599610 22270 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.629870 22284 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:12.658627 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 137532us
W20260709 12:18:12.658885 18415 connection.cc:256] RPC call timeout handler was delayed by -1.1640595540000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 12:18:12.660465 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:12.664901 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 927) took 1757 ms. Trace:
W20260709 12:18:12.661113 18415 connection.cc:256] RPC call timeout handler was delayed by -1.1378980590000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 12:18:12.636808 22338 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:12.645774 22148 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.669951 19039 rpcz_store.cc:276] 0709 12:18:10.907044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.907168 (+   124us) service_pool.cc:224] Handling call
0709 12:18:10.907448 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.564645 (+1657197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.664662 (+100017us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540020bcd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:12.664822 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713331968}
I20260709 12:18:12.627359 21818 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.656361 22330 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.632318 22334 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:18:12.654160 22349 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.691685 18293 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:18:12.695268 22344 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:18:12.696177 22245 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.698325 22270 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:12.699558 22342 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 12:18:12.699785 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66325.3us
I20260709 12:18:12.738436 18296 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.738758 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:12.738967 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43604 (request call id 693) took 1794 ms. Trace:
I20260709 12:18:12.739332 18296 rpcz_store.cc:276] 0709 12:18:10.944755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.944875 (+   120us) service_pool.cc:224] Handling call
0709 12:18:12.737062 (+1792187us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b54001ccfd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e73a3d 00007f3c6de43cd0 00007f3c673c8254 00007f3c673c81e2 00007f3c673c8111 00007f3c673c808c 00007f3c673c7342 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:12.738881 (+  1819us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9839872}
I20260709 12:18:12.739907 22331 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.702507 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.743546 18448 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:12.744880 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:12.744972 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.704535 22338 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.691390 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:12.749855 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75987.7us
I20260709 12:18:12.750573 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:12.751683 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:12.709005 22337 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.672485 22288 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.709900 22285 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.712822 21648 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:12.716074 18294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2276) took 1934 ms (client timeout 1603 ms). Trace:
I20260709 12:18:12.716847 18298 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:12.651961 22078 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:12.755811 18294 rpcz_store.cc:269] 0709 12:18:10.781985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.782143 (+   158us) service_pool.cc:224] Handling call
0709 12:18:12.715858 (+1933715us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54000726d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:12.716011 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534538496}
W20260709 12:18:12.674001 18415 connection.cc:256] RPC call timeout handler was delayed by -1.128977796000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 12:18:12.762480 18415 connection.cc:256] RPC call timeout handler was delayed by -1.0567736230000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 12:18:12.762910 18415 connection.cc:256] RPC call timeout handler was delayed by -1.0008178260000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 12:18:12.668215 22104 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:12.715971 22149 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:12.723821 22336 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.726318 22334 consensus_queue.cc:1243] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:18:12.775537 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.776173 18447 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:12.777096 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.754732 22245 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:18:12.730585 21818 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.743453 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.754640 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.748692 22248 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.783715 22285 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:12.783912 22248 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.784183 22285 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:12.784360 22248 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.785087 22285 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.786095 21968 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.786963 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.757241 22361 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:12.754492 22287 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.761571 22190 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.766050 21945 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.788738 22190 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:12.789808 22190 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.791245 22349 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:12.783315 22149 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:18:12.792137 22349 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.792511 22349 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:12.792834 22149 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:12.793239 22149 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:12.793318 22349 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.794148 22149 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.794636 22349 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.772722 22239 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.774482 22293 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.749300 22148 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.779299 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:12.781054 18448 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:12.766729 22359 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:18:12.774741 22336 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.782819 22284 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.783221 21635 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:12.787559 22252 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.778126 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:12.777643 22137 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:12.780071 22242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.486s	user 0.028s	sys 0.006s
I20260709 12:18:12.798435 18294 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:12.755667 21648 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:12.754132 22270 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.801746 18447 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:12.817656 18443 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:12.807108 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:12.810971 21818 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:12.813462 22334 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:12.819736 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:12.842931 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:12.780988 22311 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:12.845738 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:12.847664 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 98478.2us
W20260709 12:18:12.808655 22242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.515s	user 0.028s	sys 0.007s
I20260709 12:18:12.786257 22285 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.797034 22149 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.850440 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:12.863374 18294 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:12.865182 18447 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:18:12.861944 22334 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:12.817210 22252 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.774492 22253 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.754992 22341 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.871127 22331 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:12.871253 18447 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.872219 18440 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:12.872740 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:12.817649 22148 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:12.855412 22293 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.809155 21635 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:12.899348 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.908708 19042 raft_consensus.cc:2368] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:12.908882 18440 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:12.863183 22104 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:12.917635 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:12.918401 18294 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:12.919620 18294 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 10. Ops: []
W20260709 12:18:12.925333 22351 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 12:18:12.928669 18368 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:18:12.929543 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1971) took 1640 ms. Trace:
I20260709 12:18:12.808957 22284 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.856328 22270 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.929972 18368 rpcz_store.cc:276] 0709 12:18:11.288735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.927728 (+1638993us) service_pool.cc:224] Handling call
0709 12:18:12.928353 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.929222 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.929466 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.935623 22313 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
W20260709 12:18:12.937219 22360 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:18:12.938831 22324 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:12.947466 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.864924 22311 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.890594 22234 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.941431 22312 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.956463 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:12.956758 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 149704us
I20260709 12:18:12.957151 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.908965 22252 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:18:12.966161 22252 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.966567 22252 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:12.946035 22104 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:12.958804 18371 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:12.958832 18368 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.955853 22137 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.958228 22312 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:12.959738 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.984299 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.986625 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.974027 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.994774 18448 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:12.980856 18368 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:12.980932 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:12.980865 18440 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:12.988199 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:12.998860 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:13.000121 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.000847 18440 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:12.890758 21648 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.003393 18440 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:18:12.966753 22367 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:13.009814 18375 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:12.977757 22366 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:18:12.948820 22247 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.014839 22270 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.015401 22260 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:12.967461 22252 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.017607 19036 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:13.011940 18375 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:13.020036 18375 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:13.022351 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:12.976024 22334 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:12.976464 22312 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.029855 22284 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.030359 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 712) took 1696 ms. Trace:
I20260709 12:18:13.030913 22341 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.031435 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1872) took 1471 ms. Trace:
I20260709 12:18:13.043725 18298 rpcz_store.cc:276] 0709 12:18:11.559898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.565769 (+1005871us) service_pool.cc:224] Handling call
0709 12:18:12.716566 (+150797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.756490 (+ 39924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.031252 (+274762us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000726d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:13.031372 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345688320}
I20260709 12:18:13.032436 22311 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.027379 22337 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.008648 21729 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.970682 22253 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:12.969200 21722 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.972522 22104 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:13.001398 22313 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.045187 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.086418 18298 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:13.042472 18299 rpcz_store.cc:276] 0709 12:18:11.334476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.454435 (+1119959us) service_pool.cc:224] Handling call
0709 12:18:12.492882 (+ 38447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.734588 (+241706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.734618 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.734657 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:12.734691 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.030112 (+295421us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540004cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.030174 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.030297 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3562,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":555090304}
W20260709 12:18:13.075740 18297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 703) took 2098 ms (client timeout 1524 ms). Trace:
I20260709 12:18:13.087945 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:13.076011 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 697) took 2130 ms. Trace:
I20260709 12:18:12.981082 22137 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.017167 22252 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:13.084216 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.015426 22270 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.974601 22340 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.108650 18371 pending_rounds.cc:85] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 2
I20260709 12:18:13.118050 18371 pending_rounds.cc:107] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:18:13.118167 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1843) took 1716 ms. Trace:
I20260709 12:18:13.118338 18371 raft_consensus.cc:2889] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:13.087163 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.119405 18368 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:13.118698 18367 rpcz_store.cc:276] 0709 12:18:11.401654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.541240 (+139586us) service_pool.cc:224] Handling call
0709 12:18:11.541553 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.541739 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:11.562399 (+ 20660us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.074902 (+1512503us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b180042ec80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a 00007f3c68e86a2a
0709 12:18:13.117839 (+ 42937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:13.117885 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:13.117932 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.118016 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.118114 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":125,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1713,"spinlock_wait_cycles":13580160,"thread_start_us":42733,"threads_started":1}
I20260709 12:18:13.075556 21947 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.109398 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.110574 19039 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:13.027755 22331 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 12:18:13.111163 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.087051 22311 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:13.109457 18297 rpcz_store.cc:269] 0709 12:18:10.976872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.999879 (+ 23007us) service_pool.cc:224] Handling call
0709 12:18:13.075508 (+2075629us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:13.075650 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65524480}
I20260709 12:18:13.122931 18367 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:13.087026 22284 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.126187 18298 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.087715 21968 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.087598 21729 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:13.068517 22248 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:12.984630 22148 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.128068 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.071201 22280 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:13.110723 18295 rpcz_store.cc:276] 0709 12:18:10.945842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.562676 (+616834us) service_pool.cc:224] Handling call
0709 12:18:11.562983 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.666415 (+103432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.075751 (+1409336us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b54002751d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.075919 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208313088}
I20260709 12:18:13.148965 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.132339 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 711) took 1822 ms. Trace:
I20260709 12:18:13.150403 18291 rpcz_store.cc:276] 0709 12:18:11.310378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.408583 (+1098205us) service_pool.cc:224] Handling call
0709 12:18:12.408856 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.413448 (+  4592us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:12.413763 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 12:18:12.518877 (+105114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.518937 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:12.518978 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.132044 (+613066us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b54000393d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.132103 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.132266 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":357,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":31531,"spinlock_wait_cycles":25447936,"thread_start_us":104656,"threads_started":1,"wal-append.queue_time_us":237688}
I20260709 12:18:13.151748 22319 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:18:13.152403 22319 consensus_queue.cc:1243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:13.152555 21947 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.136623 22341 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.114884 22270 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.157405 22270 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.157331 21729 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.145900 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.147166 18299 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.139622 22311 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.161398 22311 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.156093 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.117600 22137 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.152431 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.150090 21635 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.159935 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.155062 21722 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.156061 18298 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:13.174065 22137 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.174801 22078 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.154760 22377 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:13.176050 22377 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.176396 22377 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:13.176846 18292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1878) took 1614 ms (client timeout 1850 ms). Trace:
W20260709 12:18:13.177266 18292 rpcz_store.cc:269] 0709 12:18:11.562042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.959740 (+397698us) service_pool.cc:224] Handling call
0709 12:18:13.176679 (+1216939us) spinlock_profiling.cc:243] Waited 80.5 ms on lock 0x7b54002369d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:13.176791 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185183104}
I20260709 12:18:13.178632 18292 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:13.169401 18295 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.179966 18296 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.155671 22341 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.184478 18298 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:13.186033 18299 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:13.171672 18297 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:13.182214 22340 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.145237 22284 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.182476 22248 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:13.143669 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 155559us
I20260709 12:18:13.187112 18291 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.129082 22104 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.189267 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:13.191821 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:13.193670 22348 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.547s	user 0.001s	sys 0.003s
I20260709 12:18:13.188225 22341 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.195869 21635 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:13.200711 22336 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.203121 18297 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:13.184445 22234 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.206743 22377 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:13.189965 22382 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:18:13.240796 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49019.6us
W20260709 12:18:13.243533 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.246330 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.248108 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.248366 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.248896 18442 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.248972 18444 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.260366 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:13.261250 18446 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:13.253885 22336 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.278483 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.266968 21722 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:13.268317 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:13.271267 22358 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.619s	user 0.004s	sys 0.002s
I20260709 12:18:13.271637 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2378) took 1361 ms. Trace:
I20260709 12:18:13.285678 18441 rpcz_store.cc:276] 0709 12:18:11.909836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.909946 (+   110us) service_pool.cc:224] Handling call
0709 12:18:11.915765 (+  5819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.983157 (+ 67392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.271401 (+1288244us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.271581 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142678784}
W20260709 12:18:13.271091 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 412442us
I20260709 12:18:13.252486 21635 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:13.257011 22104 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:13.279126 18297 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:13.287158 21635 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.287637 21635 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.248036 22317 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.281073 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:13.282269 22374 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.367s	user 0.002s	sys 0.000s
I20260709 12:18:13.293187 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.293857 18297 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:13.294772 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.296680 22248 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.297174 22248 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.283910 22336 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.298220 22336 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.298406 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.297638 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.299499 19042 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:13.299628 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.299813 19035 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:13.302059 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.303061 18412 leader_election.cc:400] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:13.303680 18412 leader_election.cc:403] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:13.303066 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:13.302033 18442 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:13.305202 18366 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.305792 18367 pending_rounds.cc:85] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:13.306234 18367 pending_rounds.cc:107] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:18:13.306556 18367 raft_consensus.cc:2889] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:13.308645 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.309403 18295 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:13.309360 18375 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.315943 18444 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:13.319195 22262 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:13.299180 22248 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.320181 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:13.320420 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:13.321115 18449 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:13.315589 22078 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:18:13.322564 18449 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:13.322948 22078 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.323370 22078 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:13.324317 22078 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:13.325491 22078 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.291399 22317 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:13.327147 18449 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.329723 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.328172 18449 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:13.328106 22317 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.333267 22317 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.332798 18449 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.334007 18449 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.319703 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:13.336059 22388 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:13.336457 22388 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.336812 22388 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.337780 18297 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:13.297685 22377 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.341850 18375 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:13.264389 22337 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:18:13.339414 22284 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.347839 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:13.365191 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.351764 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:13.354816 22239 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.357124 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 802) took 1360 ms. Trace:
I20260709 12:18:13.372241 18372 rpcz_store.cc:276] 0709 12:18:11.997148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.999312 (+  2164us) service_pool.cc:224] Handling call
0709 12:18:12.192653 (+193341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.282499 (+ 89846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.356845 (+1074346us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400395150. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:13.357048 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443975552}
I20260709 12:18:13.357889 21635 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:13.358948 22380 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 12:18:13.365701 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.377177 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.379621 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.380043 18297 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:13.383252 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.383955 18292 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:13.382714 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:13.348044 22235 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.389917 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 1968) took 2194 ms. Trace:
I20260709 12:18:13.391394 18370 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:13.391059 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.389286 18372 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:13.380702 22285 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.390341 18374 rpcz_store.cc:276] 0709 12:18:11.195944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.196283 (+   339us) service_pool.cc:224] Handling call
0709 12:18:11.234083 (+ 37800us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:11.336324 (+102241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.389667 (+2053343us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b540030ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.389864 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206263680}
I20260709 12:18:13.372012 22239 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:13.381495 22377 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.399432 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:13.348752 22322 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:18:13.390228 22363 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.395970 22033 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:13.444195 22033 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.444687 22033 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.446255 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:13.448673 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.450183 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.451864 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:13.452661 18442 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:13.453785 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:13.456852 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.458235 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.460867 22322 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.461366 22322 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:13.462181 22322 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.380993 22148 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:18:13.440295 22312 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.398841 21722 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:18:13.463872 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64552.2us
I20260709 12:18:13.465880 18444 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:13.390311 22235 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:13.477111 18412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.850s (SENT) [suppressed 42 similar messages]
I20260709 12:18:13.453749 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:13.501436 22370 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:13.518019 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.518133 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.509054 22033 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.423077 18449 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:18:13.519874 18449 pending_rounds.cc:85] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:13.520144 18449 pending_rounds.cc:107] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:18:13.520421 18449 raft_consensus.cc:2889] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:13.520754 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.520998 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.513430 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.514734 18291 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:13.523936 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.524121 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:13.464162 22293 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.024s
I20260709 12:18:13.516440 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:13.528864 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 1.10128e+06us
I20260709 12:18:13.450760 22270 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.474050 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:13.529772 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.534822 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:13.399065 22107 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:18:13.535651 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:13.536391 18338 connection.cc:256] RPC call timeout handler was delayed by -1.4712176709999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 12:18:13.539836 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.403417 22362 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:13.400578 22149 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 12:18:13.541765 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.545341 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.548122 18372 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:13.556486 19042 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:13.559625 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.568086 18366 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:13.576570 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:13.403723 22377 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:13.569054 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.415436 21947 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:18:13.522727 22235 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.631652 19042 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:18:13.632315 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.633834 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.403059 22340 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 12:18:13.438235 22363 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:13.718480 22363 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.523638 22285 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:13.724121 22284 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:13.724798 22284 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.634790 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:13.727090 18297 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:18:13.637131 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60698.1us
W20260709 12:18:13.647351 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.525307 22169 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:13.731037 22169 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.733777 22169 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.414907 22336 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.701486 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.734790 18368 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:13.735558 19041 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.735754 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.736486 18368 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:13.736861 22336 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.737200 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.738221 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.738513 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:13.522205 22370 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:13.739652 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.739854 22370 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.740723 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.741256 18368 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:18:13.742024 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.749966 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.729728 18295 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:13.702714 18374 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:13.751163 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.751494 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:13.752194 18374 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:13.752188 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:13.755784 22370 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.758283 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.412992 22253 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.221s
I20260709 12:18:13.826133 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.826713 19035 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.630559 22378 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.631433 21923 raft_consensus.cc:2764] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:18:13.767827 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.828364 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.630703 22248 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.830298 22248 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.832332 22248 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.632422 22293 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.833365 19035 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:13.834316 18446 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:13.631527 22249 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.639588 22377 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.835493 22370 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:13.773124 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.838598 19040 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:13.774883 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:13.840220 18300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 714) took 1871 ms (client timeout 2247 ms). Trace:
I20260709 12:18:13.545132 22330 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.220s
I20260709 12:18:13.651173 22281 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.782227 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:13.784227 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:13.840735 18300 rpcz_store.cc:269] 0709 12:18:11.968779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.668437 (+1699658us) service_pool.cc:224] Handling call
0709 12:18:13.840147 (+171710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:13.761864 22362 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:13.777758 22341 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
W20260709 12:18:13.788147 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 311041us
I20260709 12:18:13.800522 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:13.784111 22172 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.781697 22078 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:13.807662 18372 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:13.631111 22239 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:13.811324 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.906546 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.907143 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.847764 22330 consensus_queue.cc:1243] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:18:13.871361 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.813545 22104 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.843567 21635 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:13.874753 22311 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.000s
I20260709 12:18:13.911114 19044 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.923484 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:13.904759 22249 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.879312 22281 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:13.903533 22248 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:13.844815 22378 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.927968 22239 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.911871 22371 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.902760 22341 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.932245 22312 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.908102 22253 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:18:13.930500 22311 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.959619 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 692) took 1217 ms. Trace:
I20260709 12:18:13.962774 22404 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:13.950336 22396 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:18:13.949819 21923 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 12:18:13.965279 22405 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:13.969872 19039 rpcz_store.cc:276] 0709 12:18:12.742521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.742681 (+   160us) service_pool.cc:224] Handling call
0709 12:18:12.746291 (+  3610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.126161 (+379870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.959335 (+833174us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540071e8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:13.959547 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837210368}
I20260709 12:18:13.923406 22104 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:13.977571 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 239104us
I20260709 12:18:13.941285 22033 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.937075 22249 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.984858 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:13.964988 22378 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:13.966805 22284 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:13.909087 22172 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.977869 22270 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:13.980554 22395 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 12:18:13.984366 22398 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:18:13.993171 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:13.943248 22363 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.941715 22248 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:13.946535 22362 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.006114 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.008774 18442 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.010329 22249 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.014755 18442 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.010422 22172 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.016183 22249 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:14.017077 21923 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:14.017212 22406 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.017508 21923 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.019253 22078 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:14.019845 22078 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:14.019585 21923 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.020960 21923 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.020711 22078 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.021935 22406 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:14.024422 22406 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:14.025247 22370 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.025578 22370 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:14.026319 22370 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.953433 22371 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.994222 21635 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.994566 22407 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.978899 22404 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:13.994764 22312 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:13.998479 22378 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:14.040261 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.045914 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2290) took 2108 ms. Trace:
I20260709 12:18:14.035945 22370 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.046316 18293 rpcz_store.cc:276] 0709 12:18:11.937849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.736558 (+798709us) service_pool.cc:224] Handling call
0709 12:18:12.736815 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.765278 (+ 28463us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:12.797466 (+ 32188us) log.cc:844] Serialized 14 byte log entry
0709 12:18:12.809873 (+ 12407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.809945 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:12.809984 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.045707 (+1235723us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b5400072bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:14.045747 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.045858 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32082,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":100,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":64963072,"thread_start_us":12053,"threads_started":1,"wal-append.queue_time_us":70713,"wal-append.run_cpu_time_us":1312,"wal-append.run_wall_time_us":1056477}
I20260709 12:18:14.039216 22407 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.040804 22236 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:18:14.061597 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 808) took 2006 ms. Trace:
I20260709 12:18:14.067600 18373 rpcz_store.cc:276] 0709 12:18:12.055329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.180731 (+125402us) service_pool.cc:224] Handling call
0709 12:18:12.181060 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.653955 (+472895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.061352 (+1407397us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400414454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:14.061541 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085461248}
I20260709 12:18:14.062281 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 909) took 1335 ms. Trace:
I20260709 12:18:14.068516 22371 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.068648 22236 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:14.062471 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.992893 22281 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.070073 22363 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:14.064850 22400 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:18:14.071492 22248 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.074213 22407 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.041989 21635 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.075541 22407 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:14.055346 22388 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.000s
I20260709 12:18:14.085585 18366 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
W20260709 12:18:14.089624 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49413.9us
I20260709 12:18:14.091094 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:14.068713 18447 rpcz_store.cc:276] 0709 12:18:12.726467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.283432 (+556965us) service_pool.cc:224] Handling call
0709 12:18:13.283784 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.284092 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.284128 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.300339 (+ 16211us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.383082 (+ 82743us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:13.383134 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.383199 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.062060 (+678861us) raft_consensus.cc:1685] finished
0709 12:18:14.062083 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.062222 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5335,"raft.queue_time_us":84,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":4837,"thread_start_us":78168,"threads_started":1,"wal-append.queue_time_us":216712}
I20260709 12:18:14.093086 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:14.094379 22341 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:14.095947 22381 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.968s	user 0.005s	sys 0.000s
I20260709 12:18:14.097051 22245 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:14.058310 22312 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:18:14.108990 21635 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:14.110090 22392 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.658s	user 0.001s	sys 0.000s
W20260709 12:18:14.114348 18443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 904) took 1674 ms (client timeout 2223 ms). Trace:
W20260709 12:18:14.114841 18443 rpcz_store.cc:269] 0709 12:18:12.440161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.761243 (+321082us) service_pool.cc:224] Handling call
0709 12:18:14.114165 (+1352922us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b54000d9654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.114287 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68991360}
I20260709 12:18:14.116556 18443 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:14.117403 22404 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.049501 22378 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.124189 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2012) took 1384 ms. Trace:
I20260709 12:18:14.124588 18448 rpcz_store.cc:276] 0709 12:18:12.739360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.800029 (+ 60669us) service_pool.cc:224] Handling call
0709 12:18:14.124021 (+1323992us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400423e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.124129 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262295424}
I20260709 12:18:14.125519 22378 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.128831 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 942) took 1824 ms. Trace:
I20260709 12:18:14.143951 19036 rpcz_store.cc:276] 0709 12:18:12.304702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.328133 (+ 23431us) service_pool.cc:224] Handling call
0709 12:18:14.128594 (+1800461us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400693054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.128759 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483818240}
I20260709 12:18:14.092646 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.090976 22388 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.146926 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:14.149938 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57406us
I20260709 12:18:14.152349 21923 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:18:14.160619 21923 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:14.160720 22388 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.108872 22104 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.160995 21923 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.165063 22311 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.129779 22245 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:14.172382 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.174768 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.152870 22312 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:14.161541 22404 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:14.164228 22378 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:14.180474 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.165398 22190 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.189067 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.175799 22373 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.195729 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.196544 22239 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:14.197652 21923 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.198931 21923 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.199551 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.200539 22239 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:14.191577 22190 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.203915 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.167151 22377 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.205741 22190 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.205981 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.207403 22377 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.207096 18444 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:14.208061 22245 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.209223 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.209976 18444 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:18:14.212448 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.213325 22190 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.213797 22190 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.214207 22190 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.215546 18440 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.215793 18444 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.216251 18444 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:14.168906 22235 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.186614 22281 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.218057 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.233490 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.187590 22312 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.220108 18444 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.194779 22236 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:18:14.278524 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2000) took 1465 ms. Trace:
I20260709 12:18:14.224035 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2291) took 2286 ms. Trace:
I20260709 12:18:14.278951 18371 rpcz_store.cc:276] 0709 12:18:12.813244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.085289 (+272045us) service_pool.cc:224] Handling call
0709 12:18:13.108435 (+ 23146us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:13.131330 (+ 22895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.131376 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:13.132010 (+   634us) log.cc:844] Serialized 52 byte log entry
0709 12:18:13.448851 (+316841us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b18003b08c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:13.551329 (+102478us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:13.648601 (+ 97272us) log.cc:844] Serialized 14 byte log entry
0709 12:18:13.964584 (+315983us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b18003b08c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1
0709 12:18:14.050551 (+ 85967us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b18003b08c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:14.162901 (+112350us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b18003b08c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:14.278222 (+115321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.278291 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.278333 (+    42us) raft_consensus.cc:1685] finished
0709 12:18:14.278347 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.278471 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3583,"prepare.queue_time_us":150481,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":62526,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":19502,"spinlock_wait_cycles":29271040,"thread_start_us":217427,"threads_started":2,"wal-append.queue_time_us":629049}
I20260709 12:18:14.225455 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2395) took 1472 ms. Trace:
I20260709 12:18:14.285002 18444 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.227267 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 801) took 2230 ms. Trace:
W20260709 12:18:14.284690 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66731.2us
I20260709 12:18:14.284880 18296 rpcz_store.cc:276] 0709 12:18:11.938083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.740147 (+802064us) service_pool.cc:224] Handling call
0709 12:18:13.179791 (+439644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.180357 (+   566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.223862 (+1043505us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54002369d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.223980 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010367744}
I20260709 12:18:14.188067 22404 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.230820 18449 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:14.287217 18370 rpcz_store.cc:276] 0709 12:18:11.996826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.998885 (+  2059us) service_pool.cc:224] Handling call
0709 12:18:13.391034 (+1392149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.391644 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.227073 (+835429us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540030ff50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.227196 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3200911232}
I20260709 12:18:14.294243 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.301420 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:14.302845 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:14.242988 18375 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:14.310308 18449 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.310875 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.243111 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.248112 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.215932 22378 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.242859 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:14.284066 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.227447 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.229054 22235 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.289875 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.276015 22312 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:14.283735 21923 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:14.294227 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.324858 18374 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.294627 22340 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:14.300277 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.328338 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.236971 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.285781 18446 rpcz_store.cc:276] 0709 12:18:12.763830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.292599 (+528769us) service_pool.cc:224] Handling call
0709 12:18:13.327940 (+ 35341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.834688 (+506748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.225231 (+390543us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540035fa54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:14.225394 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163815296}
I20260709 12:18:14.329084 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:14.330312 18442 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:14.313611 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:14.309418 22311 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:14.322228 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.332350 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.285444 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.326259 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:14.316099 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.334131 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.336733 18449 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:14.337347 18444 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:18:14.337817 19038 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:14.332617 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.317175 22388 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.338663 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.340360 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.340498 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.341267 18444 pending_rounds.cc:85] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:18:14.341532 18444 pending_rounds.cc:107] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:14.342193 18444 raft_consensus.cc:2889] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:14.333262 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:14.390720 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.323120 22312 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.392426 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.324514 22408 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:14.396564 18444 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:14.397389 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.404898 18296 raft_consensus.cc:2376] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
I20260709 12:18:14.405304 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.349591 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.343602 22284 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:14.343973 22239 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:14.412104 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:14.412341 22239 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.412739 18446 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:14.412834 22239 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.413800 18446 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:14.339517 22270 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:14.415608 22284 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:14.415484 22404 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.416687 22284 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:14.401387 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.321587 22262 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:18:14.319696 21635 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:14.391680 22242 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:18:14.420748 18374 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:14.440665 18446 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:14.421916 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 72406us
I20260709 12:18:14.342171 22412 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.344787 22428 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:14.404454 22407 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:18:14.437665 18296 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:14.338307 22340 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:14.324599 21923 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.404531 22408 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:14.346993 22190 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.450266 22428 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.392683 22078 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:18:14.464362 18291 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:14.442291 22370 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:14.349864 22388 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.467557 18291 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:14.467607 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.465840 22104 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.472832 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.478047 18296 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:14.460991 21635 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:14.466058 22408 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.481827 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.485900 18293 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.466495 22262 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:14.483034 18444 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:14.488754 18296 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:14.497735 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:14.490677 22417 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.382s	user 0.000s	sys 0.004s
I20260709 12:18:14.498234 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.490489 22340 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.510321 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:14.508594 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.516590 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.519685 18291 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:14.516670 19043 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:14.521943 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:14.522797 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.481148 21635 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:14.526732 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:14.528911 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:14.530771 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190376us
I20260709 12:18:14.531404 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:14.495159 22378 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.500562 22388 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.538442 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:14.523587 22407 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.550750 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.528697 22137 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:14.555770 18447 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:18:14.556142 18447 pending_rounds.cc:85] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:18:14.556337 18447 pending_rounds.cc:107] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:14.556535 18447 raft_consensus.cc:2889] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:14.564038 22428 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.545053 22340 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:14.565248 22340 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.566174 22340 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.518054 22284 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:14.525055 22281 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:14.569425 22281 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:14.547726 22149 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:14.570574 22149 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:14.571062 22149 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.572544 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2400) took 1232 ms. Trace:
I20260709 12:18:14.573058 18440 rpcz_store.cc:276] 0709 12:18:13.339938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.543174 (+203236us) service_pool.cc:224] Handling call
0709 12:18:13.543606 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.215975 (+672369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.572244 (+356269us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:14.572472 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544862976}
I20260709 12:18:14.528683 22249 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:14.579607 18296 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:14.501274 22239 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.535070 22412 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:14.587666 18296 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:14.520181 22262 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.593695 18440 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:14.594161 18440 pending_rounds.cc:85] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 4
I20260709 12:18:14.596213 18440 pending_rounds.cc:107] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:14.596549 18440 raft_consensus.cc:2889] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:14.596817 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.586099 18293 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:14.590258 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.591861 21648 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.592465 22349 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:14.606359 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.616772 21923 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.575690 22407 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.595788 22428 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:18:14.589401 22388 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:14.633682 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:14.636315 22245 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.636996 22245 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.638448 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 830) took 1196 ms. Trace:
I20260709 12:18:14.641132 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1850) took 1082 ms. Trace:
I20260709 12:18:14.641641 18366 rpcz_store.cc:276] 0709 12:18:13.558830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.568767 (+  9937us) service_pool.cc:224] Handling call
0709 12:18:14.640880 (+1072113us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54003f2354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:14.641021 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014764800}
I20260709 12:18:14.618208 22349 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:14.633757 22078 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:14.638932 18369 rpcz_store.cc:276] 0709 12:18:13.441962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.442123 (+   161us) service_pool.cc:224] Handling call
0709 12:18:13.442470 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:13.442856 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.442882 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:13.443424 (+   542us) log.cc:844] Serialized 52 byte log entry
0709 12:18:13.784443 (+341019us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:14.637556 (+853113us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800421780. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf
0709 12:18:14.637752 (+   196us) log.cc:844] Serialized 14 byte log entry
0709 12:18:14.638011 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.638077 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.638116 (+    39us) raft_consensus.cc:1685] finished
0709 12:18:14.638222 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.638379 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":183697,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":1225,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":146,"spinlock_wait_cycles":23368960,"thread_start_us":457358,"threads_started":2,"wal-append.queue_time_us":468546}
I20260709 12:18:14.646879 22239 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.653923 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:14.654505 18440 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.658025 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.661089 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.662297 18368 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.664088 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:14.664673 18295 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.666333 22403 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:14.666721 22403 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:14.643013 22312 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.667470 22403 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.668429 22403 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.598434 22412 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.658808 18374 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:14.595312 21635 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:14.675055 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.684191 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:14.676163 18368 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:14.679666 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:14.671002 22149 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.693531 18291 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:14.696427 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:14.696530 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 158150us
I20260709 12:18:14.705542 18447 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:14.703215 18295 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:14.685266 22245 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:14.715298 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 181711us
I20260709 12:18:14.660219 22078 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.721401 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.704777 22436 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.724785 19037 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.729712 21648 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:14.725432 21923 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:14.730403 22404 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:14.744038 22404 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.746349 22404 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.723482 22283 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:18:14.752959 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.753947 18440 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:14.759383 19037 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:14.762758 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.731634 22262 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.730101 22312 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.767833 18366 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:14.760097 21648 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:14.733217 22249 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:14.768486 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84429.9us
I20260709 12:18:14.771158 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.763154 22260 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.728307 22436 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.776818 18372 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.777200 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.725687 22349 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.769119 22284 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:14.772737 22445 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.766568 22254 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.787294 22254 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:14.775696 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:14.774930 18366 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:14.781373 18369 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.782789 22445 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.793236 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:14.794971 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 134888us
I20260709 12:18:14.797253 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.776912 22262 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:14.800165 22412 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.800328 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.801350 22445 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.801736 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.809624 18369 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:14.810086 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.803280 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.810842 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.811371 18369 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.803503 18366 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:14.804643 18373 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:14.803715 22441 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.774554 22235 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:18:14.815999 22235 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:14.817230 22235 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.817328 22407 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.817762 22235 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.806792 22440 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.806845 21648 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.776032 22260 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.807771 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.780786 22268 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:14.830226 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.825906 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.831035 18373 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:14.808949 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.831182 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:14.832293 18371 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:14.837500 18294 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:14.774492 22442 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:18:14.835951 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.799108 22388 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:14.873940 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.846412 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.774677 22249 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.839640 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:14.874022 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.878901 18371 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:14.880024 22078 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.881843 22249 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.874432 22283 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:18:14.872913 22428 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.834725 22443 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:18:14.883301 22428 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.834968 22338 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.884359 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 958) took 1365 ms. Trace:
I20260709 12:18:14.875013 22388 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:14.885350 22428 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.885527 22388 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:14.878129 22404 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:14.887068 22404 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.889132 22404 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:14.876737 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:14.817224 22436 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.890487 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:14.891469 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:14.891996 18373 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:14.892388 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:14.907506 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 101201us
I20260709 12:18:14.908340 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:14.834530 22284 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:14.884841 19041 rpcz_store.cc:276] 0709 12:18:13.519174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.519326 (+   152us) service_pool.cc:224] Handling call
0709 12:18:13.541182 (+ 21856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.735961 (+194779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.884098 (+1148137us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54006bd854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:14.884290 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446571776}
I20260709 12:18:14.912827 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:14.875782 22262 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:14.912820 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 137250us
I20260709 12:18:14.919062 18369 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:14.891909 22441 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:14.927476 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50090.5us
I20260709 12:18:14.891731 22415 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.878504 22440 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.838714 22412 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:18:14.928560 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.884768 22338 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.943292 22445 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:14.952929 22423 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.729s	user 0.002s	sys 0.000s
W20260709 12:18:14.954646 22418 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.846s	user 0.002s	sys 0.000s
I20260709 12:18:14.925108 22284 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:14.936257 22285 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:18:14.956313 22284 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.960978 22408 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:14.961712 22216 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:14.963063 22216 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:14.966440 22235 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:14.967034 22248 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:14.925664 22436 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:14.967629 22248 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:14.968043 22436 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.968581 22436 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.968780 22248 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:14.970243 22248 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:14.971781 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.973066 18367 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:14.973210 22338 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:14.974385 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.975463 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:14.979573 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51089.1us
I20260709 12:18:14.966526 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:14.975433 18369 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:14.927654 22441 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.929558 21947 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:14.990347 22262 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:14.980618 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.997861 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:14.936983 22078 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.001201 22388 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.015755 18448 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:15.015930 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:15.002429 22410 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.018s	user 0.004s	sys 0.000s
W20260709 12:18:15.003159 22447 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:18:15.004176 22408 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.004648 22283 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:14.985045 22242 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:15.024665 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:15.025841 18371 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:14.985299 22445 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:15.013213 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:15.028041 18369 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:15.029479 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:15.030416 22415 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:14.994323 22441 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.035250 22262 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.035772 22445 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.041342 22452 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:15.041777 22452 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:15.044902 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.018352 22280 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:15.046574 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:15.047354 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 123349us
I20260709 12:18:15.024777 22283 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.050995 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.052049 19038 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:15.050587 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.054251 19041 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:15.056095 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:15.043893 22287 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.059713 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:15.059795 22287 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:15.027386 22242 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.027504 22442 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.074051 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.061053 19041 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:15.054183 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.060187 22281 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:15.030836 22415 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.042599 22452 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.017385 22078 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.078197 18369 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:15.046274 22408 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:15.051904 22245 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:15.061650 22283 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.063745 22280 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:15.075655 22242 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:15.078305 22415 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.062989 22312 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:18:15.160115 18299 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:15.164733 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:15.141578 21648 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:15.129120 22453 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:18:15.129906 22452 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.077850 22442 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.171646 18294 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:15.084494 21947 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:18:15.172855 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.084213 22408 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.158829 22280 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.202682 18294 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:15.173367 22428 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.224387 22428 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:15.225220 22428 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.226449 22428 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.227684 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.230661 22442 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.235718 22388 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:15.217440 22465 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:18:15.242403 22408 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:15.262024 18299 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:15.262286 22436 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:15.279212 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 369324us
W20260709 12:18:15.287632 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 213727us
I20260709 12:18:15.291764 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:15.308274 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 238469us
I20260709 12:18:15.260373 22441 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.311728 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:15.312407 22252 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:18:15.320369 22472 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:15.320860 22472 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:15.325330 22472 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.325975 22149 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.326386 22472 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:15.327543 22254 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.328228 22254 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:15.330281 22441 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.330492 22452 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.333022 22190 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.335640 18370 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:15.344177 22436 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:15.327565 22283 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.334018 22346 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.345177 22346 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:15.346064 22346 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.347118 22346 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.349977 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2410) took 1049 ms. Trace:
I20260709 12:18:15.350379 18444 rpcz_store.cc:276] 0709 12:18:14.300737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.405066 (+104329us) service_pool.cc:224] Handling call
0709 12:18:15.349824 (+944758us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b5400258ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:15.349920 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177379200}
I20260709 12:18:15.357656 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:15.362533 22283 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.311904 22415 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.351032 22242 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.359656 22149 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:18:15.382308 18295 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:15.392696 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.393226 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.396229 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.397119 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.398675 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:15.398892 19038 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [8.7-9.8]
I20260709 12:18:15.399173 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.399374 22137 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.399581 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.398890 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.399938 18296 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:15.400779 22245 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.400969 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.401368 22245 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:15.401911 19041 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:15.402406 22190 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.402326 22245 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.402966 22190 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.404313 22404 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:15.404680 22404 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:15.405364 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:15.405669 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.408056 18299 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:18:15.409142 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.409797 18293 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:15.408223 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:15.410524 19038 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:15.405068 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:15.412529 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:15.414008 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:15.414810 19038 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.414215 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.416183 18296 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:15.415930 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:15.416742 18297 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.413319 18370 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:15.418550 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:15.420769 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:15.415846 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:15.424436 18294 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.412984 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:15.434222 19043 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEARNER]: Advancing to term 9
I20260709 12:18:15.436875 18448 pending_rounds.cc:85] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:18:15.437207 18448 pending_rounds.cc:107] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:18:15.437462 18448 raft_consensus.cc:2889] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.438993 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:15.417865 22378 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:15.443799 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:15.423344 22288 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.399896 22137 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.422981 22338 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.416312 22436 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.433398 22283 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:15.449996 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:15.454265 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:15.467917 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:15.470317 18297 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:15.471611 18444 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.472474 18447 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:15.472174 18445 raft_consensus.cc:2368] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9.
I20260709 12:18:15.483098 22137 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.447427 22190 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.481810 18294 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:15.486348 19036 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:15.488260 19043 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:15.488797 22262 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.491441 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.496075 22242 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.499866 22262 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:15.501216 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.502410 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.504343 22262 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:15.504783 19037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 949) took 1744 ms (client timeout 2104 ms). Trace:
W20260709 12:18:15.505276 19037 rpcz_store.cc:269] 0709 12:18:13.760594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.763436 (+  2842us) service_pool.cc:224] Handling call
0709 12:18:15.504623 (+1741187us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b540048f8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:15.504726 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021483392}
I20260709 12:18:15.509462 22377 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.514700 22262 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.518291 22262 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:15.520327 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.514655 18194 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 212) took 9910 ms. Trace:
I20260709 12:18:15.531790 19038 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:15.540169 18294 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:15.448827 22378 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.459501 22441 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:15.437937 22445 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 12:18:15.605692 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 83639us
W20260709 12:18:15.605916 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 296717us
I20260709 12:18:15.607390 22284 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:15.607815 22270 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.608210 22270 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:15.613191 18441 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:15.614470 18441 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.614859 22260 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.616878 22190 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.617467 22190 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:15.619445 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75346.6us
I20260709 12:18:15.606554 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:15.616942 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:15.623395 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:15.623677 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:15.623960 18370 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.624233 18369 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:15.606601 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 294905us
I20260709 12:18:15.625268 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:15.558561 18444 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:15.626919 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:15.627614 18444 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
W20260709 12:18:15.616976 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.481742 22283 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.631415 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:15.632247 18293 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:15.632272 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.632329 18441 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:15.460350 22312 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.142s
I20260709 12:18:15.541225 22248 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.557520 22252 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:15.602167 22378 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:15.552407 22377 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.652138 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:15.653939 19007 leader_election.cc:400] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9.
I20260709 12:18:15.712893 19007 leader_election.cc:403] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:18:15.713488 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59730.3us
I20260709 12:18:15.472186 22338 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.599161 22415 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.604285 22281 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.661572 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.638821 22283 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:15.734230 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 72745us
I20260709 12:18:15.654461 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:15.604516 22441 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.527218 18194 rpcz_store.cc:276] 0709 12:18:05.604746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:05.604979 (+   233us) service_pool.cc:224] Handling call
0709 12:18:15.514557 (+9909578us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:10.021834 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:10.022400 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:10.022485 (+    85us) write_op.cc:435] Acquired schema lock
0709 12:18:10.022503 (+    18us) tablet.cc:662] Decoding operations
0709 12:18:10.024336 (+  1833us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:10.024448 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:10.024478 (+    30us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:18:10.281240 (+256762us) tablet.cc:701] Row locks acquired
0709 12:18:10.281277 (+    37us) write_op.cc:260] PREPARE: finished
0709 12:18:10.281449 (+   172us) write_op.cc:270] Start()
0709 12:18:10.281604 (+   155us) write_op.cc:276] Timestamp: P: 1783599490281377 usec, L: 0
0709 12:18:10.281629 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:18:10.282447 (+   818us) log.cc:844] Serialized 8963 byte log entry
0709 12:18:11.091785 (+809338us) op_driver.cc:464] REPLICATION: finished
0709 12:18:11.420435 (+328650us) write_op.cc:301] APPLY: starting
0709 12:18:11.420545 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:11.448561 (+ 28016us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:11.448717 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:11.451017 (+  2300us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:11.537945 (+ 86928us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 12:18:11.563499 (+ 25554us) write_op.cc:312] APPLY: finished
0709 12:18:11.659793 (+ 96294us) log.cc:844] Serialized 158 byte log entry
0709 12:18:11.659986 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:11.660181 (+   195us) write_op.cc:454] Released schema lock
0709 12:18:11.660263 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:11.660720 (+   457us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:11.660758 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":97959,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":286339,"apply.run_cpu_time_us":5655,"apply.run_wall_time_us":263168,"num_ops":23,"prepare.queue_time_us":151096,"prepare.run_cpu_time_us":25138,"prepare.run_wall_time_us":342372,"raft.queue_time_us":375853,"raft.run_cpu_time_us":24982,"raft.run_wall_time_us":259660,"replication_time_us":733351,"thread_start_us":497098,"threads_started":4,"wal-append.queue_time_us":102709,"wal-append.run_cpu_time_us":41287,"wal-append.run_wall_time_us":2453539}]]}
I20260709 12:18:15.678704 18369 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:15.659238 22445 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.634198 22428 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:15.631115 22388 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.740350 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.479177 22436 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.679188 18370 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:15.769480 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.684288 22477 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:18:15.561662 22404 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.660358 22248 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:15.689617 22190 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.648102 22442 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:15.719784 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67700.8us
W20260709 12:18:15.678864 22431 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.086s	user 0.012s	sys 0.003s
I20260709 12:18:15.669915 22346 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:15.739378 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85013.8us
I20260709 12:18:15.699929 22484 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.712129 22370 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.760900 18299 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:15.732465 22476 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9.
I20260709 12:18:15.768896 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:15.675729 22252 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:15.742009 22441 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:15.737701 22104 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:15.689690 22452 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:18:15.740787 22454 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.791743 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:15.732970 22377 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.741724 22281 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.777946 22260 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.748927 22472 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:18:15.854960 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63294.9us
I20260709 12:18:15.751984 22428 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.777693 22489 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:15.753171 22388 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.777637 22469 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 12:18:15.859426 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 201267us
I20260709 12:18:15.833779 22449 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:15.881803 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2023) took 1936 ms. Trace:
I20260709 12:18:15.876931 22490 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.882282 18443 rpcz_store.cc:276] 0709 12:18:13.944995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.117485 (+172490us) service_pool.cc:224] Handling call
0709 12:18:14.117860 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:14.130742 (+ 12882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:14.130769 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:14.131370 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:18:14.462608 (+331238us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:14.462650 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.462749 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.619007 (+1156258us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:15.876465 (+257458us) raft_consensus.cc:1685] finished
0709 12:18:15.876492 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.881724 (+  5232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14865,"raft.queue_time_us":24843,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6337,"spinlock_wait_cycles":4606720,"thread_start_us":287363,"threads_started":1,"wal-append.queue_time_us":378471}
I20260709 12:18:15.893410 22490 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.861197 22104 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:15.863720 22489 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.803211 22370 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.862284 22363 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:15.800639 22484 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.861335 22346 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:18:15.898015 22487 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:15.929108 22377 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.930884 22388 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:15.932988 22492 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:15.933176 22484 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.933867 22484 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.934398 22484 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.936781 22370 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.937517 22370 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.940181 22436 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.940853 22493 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:15.941357 22493 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:15.944542 22428 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.947567 22104 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.948632 22338 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.913944 22498 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:15.950294 22498 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.952894 22498 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.960845 22287 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.962283 22287 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.962401 22248 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.902026 22489 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.961575 22281 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:15.901304 22284 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:18:15.967698 22487 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:15.968675 22487 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.900302 22252 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:15.921878 22483 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:18:15.942365 21635 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.988660 22078 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.942375 22493 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:15.912688 22363 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:15.998699 22436 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:15.999372 22436 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.001677 22436 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:16.008630 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.010913 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.009253 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 231880us
I20260709 12:18:16.033851 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:16.024778 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 255321us
W20260709 12:18:16.126333 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 89492.2us
W20260709 12:18:16.126726 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.129891 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 349449us
I20260709 12:18:16.023679 22490 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:15.997089 22428 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:18:16.134693 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 332120us
I20260709 12:18:16.154810 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:16.132894 22234 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:16.172055 22234 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.172573 22234 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:16.174280 19043 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [8.7-9.8]
I20260709 12:18:16.175081 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.179833 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:16.180307 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.153800 22472 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:18:16.181710 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.182673 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.185865 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.188360 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:16.160048 22285 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.191092 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.154794 22442 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:16.191272 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.201862 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:16.202312 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.160964 22428 consensus_queue.cc:1243] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:16.206056 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.154353 21635 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:16.197844 19044 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:16.199123 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.209878 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.211437 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.207067 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.204514 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.215636 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:16.216043 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:16.216382 18369 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:16.216745 18370 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:16.184119 22078 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.153452 22500 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:18:16.223178 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:16.223829 18371 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:16.217890 19044 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:16.170940 22445 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:16.218910 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:16.220420 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:16.221611 18373 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:16.222333 18367 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.185039 22235 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:18:16.236037 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.236685 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.236821 18299 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 8: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:16.227036 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.003s	sys 0.010s
W20260709 12:18:16.227331 18446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.007s
I20260709 12:18:16.199162 22441 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:16.245759 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.232492 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:16.208719 22251 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.233269 19040 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:16.225836 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:16.232954 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.239457 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.251936 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:16.239444 18446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.006s	sys 0.007s
W20260709 12:18:16.226815 18448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.000s	sys 0.013s
W20260709 12:18:16.253119 18448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.001s	sys 0.014s
I20260709 12:18:16.240547 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.253847 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:16.226914 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.003s	sys 0.010s
I20260709 12:18:16.219564 22248 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.247754 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:16.254866 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.003s	sys 0.010s
I20260709 12:18:16.250746 19042 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:16.254011 18293 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
W20260709 12:18:16.255267 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 843506us
I20260709 12:18:16.244367 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:16.255280 18295 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:16.258299 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:16.240010 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.260025 18299 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:16.268795 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:16.247395 22288 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.002s
I20260709 12:18:16.207297 22285 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:16.272647 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.217180 21635 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.244825 22484 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:16.220966 22378 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.285728 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:16.271502 18368 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:16.248304 22388 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:16.273429 22490 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.292981 18367 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:18:16.309847 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 120765us
W20260709 12:18:16.311421 22283 consensus_meta.cc:287] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.005s
I20260709 12:18:16.322232 19042 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:16.320199 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:16.286542 22288 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.281113 22251 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.310925 22484 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:16.308414 22253 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.288542 22507 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:18:16.309371 22245 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:16.309635 22442 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.348922 18300 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:16.349834 22513 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:16.386667 22513 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:16.387084 22513 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.393276 22195 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:16.393710 22195 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:16.334790 22248 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:18:16.396066 22245 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.368530 22285 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:16.369933 21635 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:16.370126 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:16.398070 22482 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.797s	user 0.002s	sys 0.001s
I20260709 12:18:16.398201 19042 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:16.333305 22078 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:16.309546 22505 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:18:16.399658 22497 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:18:16.375025 22378 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:16.400650 22281 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:16.381019 22489 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.346405 22251 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:16.346276 22484 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:16.384789 22480 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.902s	user 0.004s	sys 0.000s
I20260709 12:18:16.403898 19039 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:16.404640 22489 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:16.404065 22484 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.396569 22245 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:16.406028 22245 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.330826 22283 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.407186 22245 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.409117 22283 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:16.332110 22363 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:16.328960 22516 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.316171 22490 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.311933 22388 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.370673 22476 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:16.370687 22441 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:16.371790 22312 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:18:16.341606 22253 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.394531 22195 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.411270 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.394754 22513 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:16.396567 22479 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.678s	user 0.013s	sys 0.000s
I20260709 12:18:16.402460 22251 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.365955 22370 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:16.443245 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:16.445418 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.462059 18296 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:16.446808 18294 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:16.447922 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.463333 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.464298 18294 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:16.434067 22476 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:16.433135 22516 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:16.481530 18296 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:16.844687 22338 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.608886 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 289190us
I20260709 12:18:16.514272 22195 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.847978 22476 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:16.737555 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 451829us
I20260709 12:18:16.848440 22338 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.850229 22476 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:16.454617 22407 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.590065 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.584707 21648 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.268s
I20260709 12:18:16.502341 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.852555 22281 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:16.854000 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.854367 22516 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:16.854734 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:16.502534 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.852468 22407 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.855744 18297 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:16.691651 22370 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.856711 22370 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:16.858529 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.593739 22288 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.503216 18292 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:16.663002 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.548539 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.861842 22253 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:16.691696 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 398589us
I20260709 12:18:16.871131 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:16.588912 22234 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:16.847186 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.873775 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.863654 22516 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:16.828013 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:16.869024 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:16.862370 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:16.878167 18294 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:16.875874 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:16.849411 18296 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:16.889045 18292 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:16.905640 19043 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:16.901896 19036 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:16.907490 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:16.872903 22370 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:16.870194 22281 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.895478 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 176507us
I20260709 12:18:16.904695 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:16.920394 22521 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
I20260709 12:18:16.880183 22505 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.891417 22378 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:16.931349 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.880580 22516 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:16.920917 22510 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:16.930284 22370 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.893151 22514 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.917009 22287 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:16.880613 22288 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:16.909874 22452 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:16.890600 22454 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:18:16.958245 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.942600 22516 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:16.926383 22281 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.028277 22504 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.028750 22281 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.028956 22504 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:17.033383 22522 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:18:17.034502 22436 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:17.036099 22436 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:17.036639 22436 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:17.038523 22507 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.041400 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 938) took 1429 ms. Trace:
W20260709 12:18:17.046416 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.953917 22514 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.047561 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:17.048081 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:17.048695 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:17.049664 22415 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:17.049786 22514 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.049968 22284 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:17.050215 22415 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:17.050669 22284 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.051097 22415 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.052194 21635 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.052868 22311 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:17.053169 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:17.053751 18441 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.053454 22311 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.055642 18297 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:17.055820 18442 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:17.033416 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 936) took 1431 ms. Trace:
I20260709 12:18:17.057024 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:17.049834 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:17.058452 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.041843 18449 rpcz_store.cc:276] 0709 12:18:15.611920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.615311 (+  3391us) service_pool.cc:224] Handling call
0709 12:18:15.615551 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.874651 (+259100us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:15.903588 (+ 28937us) log.cc:844] Serialized 14 byte log entry
0709 12:18:16.171294 (+267706us) log.cc:844] Serialized 14 byte log entry
0709 12:18:16.171448 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:16.171500 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:16.171562 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.041159 (+869597us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54001a6154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:17.041204 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.041346 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9418,"prepare.queue_time_us":46502,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":6915,"raft.queue_time_us":133951,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":10986,"spinlock_wait_cycles":603375872,"thread_start_us":259131,"threads_started":1,"wal-append.queue_time_us":447935}
I20260709 12:18:17.059805 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.060608 18444 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:17.057088 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.056960 18447 rpcz_store.cc:276] 0709 12:18:15.601530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.601710 (+   180us) service_pool.cc:224] Handling call
0709 12:18:15.605838 (+  4128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.797759 (+191921us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:16.167745 (+369986us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.032166 (+864421us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.032238 (+    72us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:17.032283 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.033183 (+   900us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400375854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:17.033228 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.033359 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":688057,"prepare.queue_time_us":112174,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":694267,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":440887040,"thread_start_us":491829,"threads_started":2,"wal-append.queue_time_us":572145}
I20260709 12:18:16.947736 22253 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.079968 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.080709 18447 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:17.078117 18295 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:17.079597 22137 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:17.077275 18444 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:17.084672 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.085407 18444 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 7.
I20260709 12:18:17.086470 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.037588 22436 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.087628 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.052700 22284 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.094729 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.096858 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.097198 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:17.097775 18447 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:17.055106 22534 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:17.006018 22452 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:16.934144 22505 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.018522 22388 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.106134 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:17.075027 22370 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:17.080026 22513 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.007380 22287 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.039108 22507 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.087822 22441 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.111174 18367 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:17.242954 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1876) took 1021 ms. Trace:
I20260709 12:18:17.112109 18373 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [9.10-12.12]
I20260709 12:18:17.113014 18369 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:17.115023 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.243458 18368 rpcz_store.cc:276] 0709 12:18:16.221246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.221349 (+   103us) service_pool.cc:224] Handling call
0709 12:18:16.229167 (+  7818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.229405 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:16.328171 (+ 98766us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.104685 (+776514us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.242535 (+137850us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.242602 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:17.242671 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.242759 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.242903 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137599,"raft.queue_time_us":8114,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":497872,"threads_started":2,"wal-append.queue_time_us":620802}
I20260709 12:18:17.244735 18448 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:17.112460 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.095329 22137 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:17.263669 19044 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:17.052734 21635 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.088055 22442 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:17.284842 18441 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:17.095116 22523 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:17.238969 22526 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:17.102151 22514 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:17.247778 18369 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:18:17.232878 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 126837us
I20260709 12:18:17.087548 22408 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.143s
I20260709 12:18:17.242971 22285 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:17.248271 22281 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:17.248817 22078 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:17.250268 22404 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.250547 18366 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 12:18:17.251695 18371 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:17.252980 22252 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:17.253191 22531 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 12:18:17.254159 22535 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:18:17.224009 22284 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:18:17.255473 22234 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.256322 22288 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.258061 22370 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.232162 21923 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:17.260175 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:17.381237 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 486250us
I20260709 12:18:17.384153 19039 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:17.264353 22441 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.264140 22137 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.272565 22287 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.091981 22253 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.273564 22452 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.074043 22533 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:17.100456 22484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.086s	sys 0.107s
I20260709 12:18:17.302213 22280 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.288841 18367 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.290011 18447 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:17.291874 18442 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:17.288776 18441 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.246146 22311 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.246047 18370 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:17.309223 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:17.228431 22377 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:18:17.245965 22504 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.295178 22454 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:18:17.285889 22262 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:17.360312 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.115289 22534 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.284688 22190 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.386703 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:17.298280 22513 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:17.101613 22245 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.694s	user 0.006s	sys 0.006s
I20260709 12:18:17.320827 22503 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.295460 22442 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.319977 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:17.388043 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:17.406021 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 96921.8us
I20260709 12:18:17.389806 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:17.376340 22281 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:17.407898 18340 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.016s
I20260709 12:18:17.516012 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:17.516485 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.005s	sys 0.016s
W20260709 12:18:17.517309 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.375643 22312 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:17.412936 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.527462 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.435052 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.527563 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 438460us
W20260709 12:18:17.396858 22484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.087s	sys 0.111s
W20260709 12:18:17.449385 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:17.467986 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 81388.8us
I20260709 12:18:17.372711 22514 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.579475 22514 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.378024 22078 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.581331 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.407100 18373 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:17.581969 18372 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:17.581985 22078 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.583369 22288 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:17.582947 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.583843 22288 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.584149 18372 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:17.519834 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.584903 18370 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:17.585842 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.521065 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.587525 18367 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:17.421536 22287 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.588405 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.452762 22377 consensus_queue.cc:1243] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:17.536684 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:17.518460 22531 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:18:17.542317 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:17.548612 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:17.550900 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:17.605472 18296 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:18:17.605576 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 54795.9us
I20260709 12:18:17.606426 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:17.608121 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.609297 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.326730 22526 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.531606 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:17.612331 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 80813.5us
I20260709 12:18:17.575503 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:17.544196 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:17.409374 22280 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.586923 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:17.613147 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:17.614800 18370 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:17.615578 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:17.518172 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:17.616597 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 98511.4us
I20260709 12:18:17.617203 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:17.617695 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:17.586385 18366 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 6.
I20260709 12:18:17.396559 21923 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.583838 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.597620 19039 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:17.539683 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:17.610414 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.620400 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:17.621114 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.621702 18441 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:17.622382 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.608147 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:17.584282 22288 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.624178 18372 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:17.625022 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:17.412009 22533 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.619357 18373 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:17.594420 22235 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.629699 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:17.594998 22287 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:17.598294 22415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.004s	sys 0.035s
I20260709 12:18:17.638159 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:17.648376 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.648597 18441 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:17.610553 22078 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:17.630439 18449 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.008s	sys 0.016s
I20260709 12:18:17.649776 18293 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:17.511020 22190 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.514081 22513 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.507555 22262 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.532049 22285 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
W20260709 12:18:17.655220 18449 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.008s	sys 0.016s
I20260709 12:18:17.656134 18370 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:17.657236 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.631634 22242 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:17.618413 22284 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.587074 22312 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:17.625109 22487 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.024s	sys 0.028s
I20260709 12:18:17.598811 22408 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:18:17.626533 22245 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:17.666529 18446 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.016s	sys 0.035s
I20260709 12:18:17.626649 22377 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:17.630970 22484 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.653842 22445 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:17.653872 22476 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.803s	user 0.004s	sys 0.007s
W20260709 12:18:17.671690 18446 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.016s	sys 0.035s
I20260709 12:18:17.673439 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:17.647534 22287 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.674449 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:17.647810 21635 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.528156 22503 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.688153 18369 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:17.654886 22540 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.670298 22415 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:17.665561 22242 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.659479 22505 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.667301 22284 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:17.670810 22487 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.024s	sys 0.029s
W20260709 12:18:17.688551 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.009s	sys 0.050s
W20260709 12:18:17.703186 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.050s
I20260709 12:18:17.689067 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:17.676705 22245 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.658689 22504 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:17.681401 22484 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:17.671001 18413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.012s	sys 0.017s
W20260709 12:18:17.813813 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45063.1us
W20260709 12:18:17.823494 18413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.012s	sys 0.017s
I20260709 12:18:17.655545 22235 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.677120 22541 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.695148 22242 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:17.696386 22415 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.695612 22540 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.709290 22284 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.688939 22078 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.688375 22445 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.829761 22547 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.855769 22547 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.858080 22547 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.858704 22489 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.859920 22551 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.860591 22551 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.861070 22555 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:17.862033 22555 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.862460 22555 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.862768 22555 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.863421 22547 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:17.860589 22536 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 12:18:17.865046 22235 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.865103 22547 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:17.866853 22190 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.867616 22547 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.868206 22547 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:17.863109 22551 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.869158 22551 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.869804 22503 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.869674 22551 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.720805 21635 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.692193 22377 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:17.741636 22245 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.879908 22533 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.881515 22407 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:17.882644 22537 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.460s	user 0.000s	sys 0.001s
I20260709 12:18:17.855008 22445 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.815703 22484 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.843495 22415 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:17.720606 22476 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.883270 22459 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:17.835041 22541 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:17.910048 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 443796us
I20260709 12:18:17.960003 18370 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:17.725314 18369 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:17.721020 22507 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.845233 22441 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:17.843294 22540 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.889045 22510 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:17.972846 18368 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.017s	sys 0.033s
W20260709 12:18:17.973209 18368 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.017s	sys 0.033s
I20260709 12:18:17.966996 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.967149 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:17.889941 22253 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:17.984053 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.899601 22408 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:17.984812 18443 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:17.994422 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:17.994864 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.995690 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:17.996351 18443 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:17.949839 22533 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:17.957778 22459 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:17.890955 22489 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.892647 22078 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:17.961143 22541 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.968598 22534 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.966482 22503 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:18.029053 22470 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.036494 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:18.074880 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.075336 18443 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 7: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:17.972147 22510 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:17.984155 22557 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.905154 22547 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.051146 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.037628 22281 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:17.970227 22476 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.106793 22476 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:18.093786 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.903249 21635 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:18.094275 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 477093us
I20260709 12:18:18.132849 21635 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.050834 21923 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.012650 22408 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.134845 18297 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 12:18:18.135066 22408 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.073094 22235 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.135872 22235 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.136567 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.138018 22235 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.107967 22579 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:18.138737 22579 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:18.139753 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.140831 22489 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.140985 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.141902 18297 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
W20260709 12:18:18.142269 22556 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:18:18.143013 22078 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.112865 22280 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.035902 22533 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:18.144119 22510 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.146593 22544 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:18.146899 22533 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.114194 22487 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:18.116389 18295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 741) took 1672 ms (client timeout 1575 ms). Trace:
W20260709 12:18:18.148284 18295 rpcz_store.cc:269] 0709 12:18:16.444229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.444337 (+   108us) service_pool.cc:224] Handling call
0709 12:18:18.116108 (+1671771us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54001cc0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.116295 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509945728}
I20260709 12:18:18.148463 22459 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.150097 22459 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:18.152943 22547 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.097070 22557 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.156631 22557 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.156560 22541 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:18.158695 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 749) took 1700 ms. Trace:
I20260709 12:18:18.159137 18299 rpcz_store.cc:276] 0709 12:18:16.458866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.480406 (+ 21540us) service_pool.cc:224] Handling call
0709 12:18:16.844999 (+364593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.845614 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:17.296147 (+450533us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.051574 (+755427us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.080034 (+ 28460us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.080244 (+   210us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:18.080300 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.080398 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.158451 (+ 78053us) raft_consensus.cc:1685] finished
0709 12:18:18.158470 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.158628 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20316,"prepare.queue_time_us":761882,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":5450,"raft.queue_time_us":233816,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":18999,"spinlock_wait_cycles":24034176,"thread_start_us":1205158,"threads_started":3,"wal-append.queue_time_us":548276}
I20260709 12:18:18.161854 22284 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.162735 22557 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.163460 22557 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.166114 22557 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.168947 22541 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.172271 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 747) took 1724 ms. Trace:
I20260709 12:18:18.172827 18298 rpcz_store.cc:276] 0709 12:18:16.448121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.448248 (+   127us) service_pool.cc:224] Handling call
0709 12:18:16.448683 (+   435us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:16.514003 (+ 65320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.514050 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:16.607838 (+ 93788us) log.cc:844] Serialized 78 byte log entry
0709 12:18:16.943460 (+335622us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:16.943502 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.943573 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.171957 (+1228384us) raft_consensus.cc:1685] finished
0709 12:18:18.171988 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.172195 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5954,"raft.queue_time_us":44651,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":79542,"thread_start_us":146509,"threads_started":1,"wal-append.queue_time_us":577352}
I20260709 12:18:18.051209 22551 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.070773 22377 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.103080 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.189627 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:18.190822 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.191572 18443 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:18.120518 22513 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.121742 22493 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:18.123945 22550 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:18:18.124872 22338 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.119410 22270 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.215026 22484 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.091810 22470 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.218663 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 947) took 1166 ms. Trace:
I20260709 12:18:18.219000 18445 rpcz_store.cc:276] 0709 12:18:17.052191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.057291 (+  5100us) service_pool.cc:224] Handling call
0709 12:18:17.057578 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.057882 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.057906 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.081434 (+ 23528us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.631162 (+549728us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:17.631200 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.632241 (+  1041us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.218445 (+586204us) raft_consensus.cc:1685] finished
0709 12:18:18.218468 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.218606 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30628,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9746,"thread_start_us":403874,"threads_started":1,"wal-append.queue_time_us":729545}
I20260709 12:18:18.221561 18445 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:18.225448 22484 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:18.198876 22252 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.823s	user 0.006s	sys 0.005s
I20260709 12:18:18.228513 18440 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:18.084017 22503 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.266438 22523 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.934s	user 0.009s	sys 0.001s
W20260709 12:18:18.266552 22288 consensus_meta.cc:287] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.000s
I20260709 12:18:18.267192 22288 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:18.268784 22516 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.187s	user 0.006s	sys 0.006s
I20260709 12:18:18.274050 22516 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.267019 22523 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:18.291491 22540 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.307857 22555 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:18.351089 18266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.007s	sys 0.015s
I20260709 12:18:18.143040 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:18.389290 18266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.009s	sys 0.015s
I20260709 12:18:18.353861 18373 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:18.389739 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 750873us
W20260709 12:18:18.359447 18446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.016s	sys 0.023s
W20260709 12:18:18.390903 18446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.017s	sys 0.023s
W20260709 12:18:18.360193 18374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.068s	sys 0.099s
W20260709 12:18:18.392462 18374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.068s	sys 0.100s
I20260709 12:18:18.394831 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 843) took 1257 ms. Trace:
I20260709 12:18:18.365829 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 846) took 1117 ms. Trace:
I20260709 12:18:18.370100 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2037) took 1316 ms. Trace:
I20260709 12:18:18.372226 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2047) took 1288 ms. Trace:
I20260709 12:18:18.377058 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 850) took 1084 ms. Trace:
W20260709 12:18:18.384207 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2041) took 1329 ms (client timeout 1655 ms). Trace:
W20260709 12:18:18.384097 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 789507us
W20260709 12:18:18.360405 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.399417 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.399452 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 322253us
I20260709 12:18:18.543172 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 944) took 1480 ms. Trace:
W20260709 12:18:18.272276 22404 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.001s
I20260709 12:18:18.582340 22470 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.580888 18372 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:18.582435 22577 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.396456 18375 rpcz_store.cc:276] 0709 12:18:17.292992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.293134 (+   142us) service_pool.cc:224] Handling call
0709 12:18:17.321366 (+ 28232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.602534 (+281168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.602568 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.602626 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:17.602665 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.376362 (+773697us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400395654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.376450 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.376825 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646083328}
W20260709 12:18:18.396978 18440 rpcz_store.cc:269] 0709 12:18:17.054610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.056751 (+  2141us) service_pool.cc:224] Handling call
0709 12:18:18.383552 (+1326801us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002049d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.383943 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691076352}
W20260709 12:18:18.586126 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.653s (SENT) [suppressed 7 similar messages]
I20260709 12:18:18.396160 18447 rpcz_store.cc:276] 0709 12:18:17.084125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.111467 (+ 27342us) service_pool.cc:224] Handling call
0709 12:18:17.285824 (+174357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.401457 (+115633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.371630 (+970173us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b5400202bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.371972 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139480960}
I20260709 12:18:18.395810 18442 rpcz_store.cc:276] 0709 12:18:17.053503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.056785 (+  3282us) service_pool.cc:224] Handling call
0709 12:18:18.369493 (+1312708us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54000d9b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.369767 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505106816}
I20260709 12:18:18.472219 22532 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:18.471633 22530 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:18.587936 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 187393us
I20260709 12:18:18.594762 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.598873 18366 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:18.575031 18446 rpcz_store.cc:276] 0709 12:18:17.051288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.055609 (+  4321us) service_pool.cc:224] Handling call
0709 12:18:17.094787 (+ 39178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.685332 (+590545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.685375 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.693416 (+  8041us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.391760 (+698344us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:18.391882 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.392060 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.409273 (+ 17213us) raft_consensus.cc:1685] finished
0709 12:18:18.409294 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.532058 (+122764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10472,"raft.queue_time_us":183745,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9613,"spinlock_wait_cycles":52352,"thread_start_us":1139154,"threads_started":2,"wal-append.queue_time_us":685900}
I20260709 12:18:18.580005 22540 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.605293 22503 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.603731 18297 raft_consensus.cc:2368] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8.
I20260709 12:18:18.605620 22577 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.594702 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.594496 22555 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.616904 22516 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:18.619112 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59827.7us
I20260709 12:18:18.621322 21648 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:18.625831 22571 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:18.626724 22587 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:18.627852 18299 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:18.630159 22587 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:18.630729 22587 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:18.631453 22283 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.632254 22283 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.632401 22587 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.634611 22587 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.638216 18296 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:18.560628 22262 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.396322 18374 rpcz_store.cc:276] 0709 12:18:17.137235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.137383 (+   148us) service_pool.cc:224] Handling call
0709 12:18:17.137733 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.137965 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:17.704741 (+566776us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.393909 (+689168us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.394248 (+   339us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.394321 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:18.394402 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.394514 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.394750 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":421182,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":15644,"raft.queue_time_us":211899,"raft.run_cpu_time_us":522,"raft.run_wall_time_us":18847,"thread_start_us":1165559,"threads_started":2,"wal-append.queue_time_us":698918}
I20260709 12:18:18.619933 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.395691 18368 rpcz_store.cc:276] 0709 12:18:17.248523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.275407 (+ 26884us) service_pool.cc:224] Handling call
0709 12:18:17.275780 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.276071 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:17.973654 (+697583us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.364658 (+391004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.364796 (+   138us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:18.364988 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.365186 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.365503 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":332794,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":33439,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":52355,"thread_start_us":1087363,"threads_started":2,"wal-append.queue_time_us":442041}
I20260709 12:18:18.649268 22280 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:18:18.654433 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.657721 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.658476 18292 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:18.658373 22338 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:18.658512 18298 raft_consensus.cc:2410] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:18.598846 22530 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.664074 22280 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:18.665050 22280 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:18.665112 22338 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:18.665817 22338 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:18.661790 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:18.659616 22284 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.666705 22280 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.676028 22542 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.827s	user 0.003s	sys 0.000s
I20260709 12:18:18.677186 22459 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:18:18.684628 22490 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.677668 22584 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.669122 22338 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.701390 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:18.694593 22571 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:18.703888 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.706051 19043 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:18.705384 22571 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.711316 22577 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.711619 22571 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.710848 22338 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.719532 22577 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.719995 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:18.721235 22490 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:18.721614 22490 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.721961 22490 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:18.720049 22577 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:18.723052 22577 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.725620 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.739387 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.740365 19043 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 7.
W20260709 12:18:18.743896 22415 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.775s	user 0.005s	sys 0.012s
W20260709 12:18:18.752750 22500 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.008s	sys 0.000s
W20260709 12:18:18.753304 22500 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.009s	sys 0.000s
W20260709 12:18:18.758309 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.761413 22280 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.761662 22532 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:18.762238 22532 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.768352 22530 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.769685 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 748) took 2320 ms. Trace:
W20260709 12:18:18.770560 22585 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:18:18.757755 22415 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.771512 22526 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:18.771620 22242 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.876s	user 0.002s	sys 0.024s
W20260709 12:18:18.775844 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56001.7us
W20260709 12:18:18.783895 22287 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.089s	user 0.015s	sys 0.003s
I20260709 12:18:18.784351 22287 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.785753 18295 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:18.786542 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:18.776965 18291 rpcz_store.cc:276] 0709 12:18:16.449409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.449537 (+   128us) service_pool.cc:224] Handling call
0709 12:18:16.829366 (+379829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.906767 (+ 77401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.906799 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.907283 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.023939 (+116656us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b1800388c00. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:17.510902 (+486963us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:17.883935 (+373033us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.916024 (+ 32089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.114528 (+198504us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.114577 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.658987 (+544410us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b1800388c00. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db
0709 12:18:18.665976 (+  6989us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:18.769217 (+103241us) raft_consensus.cc:1685] finished
0709 12:18:18.769238 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.769459 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52392,"prepare.queue_time_us":127113,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":4372,"raft.queue_time_us":75437,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":16766,"spinlock_wait_cycles":193783040,"thread_start_us":777924,"threads_started":2,"wal-append.queue_time_us":766949}
I20260709 12:18:18.749135 22584 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.790858 22378 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.791954 22584 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.791651 22378 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.792485 22312 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.794168 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:18.795392 22137 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:18.795617 22283 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.795825 22137 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:18.796288 22283 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:18.797267 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:18.799823 22312 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:18.800232 22312 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.800592 22312 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:18.802148 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2436) took 1149 ms. Trace:
I20260709 12:18:18.802558 18441 rpcz_store.cc:276] 0709 12:18:17.652799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.656948 (+  4149us) service_pool.cc:224] Handling call
0709 12:18:18.801795 (+1144847us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.801961 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137772416}
I20260709 12:18:18.772289 22288 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.803702 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.804050 18300 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 11. Ops: []
I20260709 12:18:18.805056 22415 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:18.806763 22493 consensus_meta.cc:287] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.009s
I20260709 12:18:18.808048 18371 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:18:18.808487 18371 pending_rounds.cc:85] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:18.808801 18371 pending_rounds.cc:107] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:18:18.809088 18371 raft_consensus.cc:2889] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:18.810811 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.813313 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2044) took 1730 ms. Trace:
I20260709 12:18:18.813401 18374 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:18.814649 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.815637 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.815761 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.816421 19036 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:18.816965 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.817344 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.818392 19037 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:18.819378 18446 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:18.819721 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.820101 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2038) took 1766 ms. Trace:
I20260709 12:18:18.819918 22242 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.820518 18449 rpcz_store.cc:276] 0709 12:18:17.053794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.059525 (+  5731us) service_pool.cc:224] Handling call
0709 12:18:17.059876 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:17.662493 (+602617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.662535 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:17.663197 (+   662us) log.cc:844] Serialized 52 byte log entry
0709 12:18:18.049922 (+386725us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:18.084557 (+ 34635us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.819613 (+735056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.819735 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.819801 (+    66us) raft_consensus.cc:1685] finished
0709 12:18:18.819817 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.819933 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":763874,"prepare.queue_time_us":39911,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":185167,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10111,"spinlock_wait_cycles":46848,"thread_start_us":931107,"threads_started":2,"wal-append.queue_time_us":511559}
I20260709 12:18:18.812227 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:18.824090 18440 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.811452 22540 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.811902 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:18.813750 18444 rpcz_store.cc:276] 0709 12:18:17.082999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.098353 (+ 15354us) service_pool.cc:224] Handling call
0709 12:18:17.098680 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.098906 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:17.674553 (+575647us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.812702 (+1138149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.812798 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:18.812861 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.812960 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.813133 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":777148,"prepare.queue_time_us":785520,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"thread_start_us":809053,"threads_started":2,"wal-append.queue_time_us":536919}
W20260709 12:18:18.822079 22245 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.871s	user 0.016s	sys 0.008s
I20260709 12:18:18.796334 22137 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.826817 22104 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:18.790791 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:18.824316 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:18.822259 22242 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:18.842188 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.841925 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:18.843009 19036 raft_consensus.cc:606] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:18.843549 19036 raft_consensus.cc:614] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:18.819612 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:18.851163 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.827548 22104 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:18.853579 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63092.5us
I20260709 12:18:18.854173 22252 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:18.855186 22104 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.855531 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:18.855859 22104 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:18.856941 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:18.855937 22408 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:18.857867 22408 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.858901 22445 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.859490 22445 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:18.859615 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.860001 22445 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.860432 18372 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:18.860833 22408 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.854895 22252 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.861728 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.862705 18368 raft_consensus.cc:2410] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:18.856716 22104 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.851006 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:18.839771 22503 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.863313 22252 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.882061 18374 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:18.881013 22484 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:18.883723 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53058.3us
I20260709 12:18:18.885039 22370 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:18.885635 22370 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:18.886265 22370 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.887769 22104 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.833969 22540 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:18.890844 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.891541 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:18.892836 22251 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.893232 22251 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:18.893591 22251 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.883354 22493 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:18.896411 22493 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:18.830952 22245 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.896845 22493 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.896976 22234 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:18.897575 22234 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:18.898514 22245 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.901080 22287 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.901393 22378 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.903600 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2435) took 1251 ms. Trace:
I20260709 12:18:18.904150 18447 rpcz_store.cc:276] 0709 12:18:17.652554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.601905 (+949351us) service_pool.cc:224] Handling call
0709 12:18:18.648784 (+ 46879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.903004 (+254220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.903056 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.903151 (+    95us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:18.903200 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.903337 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.903493 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":220038}
I20260709 12:18:18.907514 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 752) took 1325 ms. Trace:
I20260709 12:18:18.907893 18293 rpcz_store.cc:276] 0709 12:18:17.582412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.585029 (+  2617us) service_pool.cc:224] Handling call
0709 12:18:17.585405 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.654887 (+ 69482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.906978 (+1252091us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b54001cc5d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.907325 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147362560}
I20260709 12:18:18.909924 22442 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.914417 22452 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:18.915457 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:18.918028 18440 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:18.919165 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:18.919821 19036 raft_consensus.cc:606] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:18.921113 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:18.932335 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.898886 22234 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.924213 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:18.935614 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 124079us
W20260709 12:18:18.937480 22582 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:18:18.937850 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:18.937536 22403 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:18.938824 18368 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 12:18:18.941686 22546 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.061s	user 0.003s	sys 0.001s
I20260709 12:18:18.926040 22540 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.954661 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 853) took 1580 ms. Trace:
I20260709 12:18:18.955322 22601 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:18.948846 22378 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:18.951591 22442 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:18.958060 22601 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.958595 22601 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:18.955168 18371 rpcz_store.cc:276] 0709 12:18:17.374650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.383450 (+  8800us) service_pool.cc:224] Handling call
0709 12:18:17.383850 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.809652 (+1425802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.943669 (+134017us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400394c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.954591 (+ 10922us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3273819520}
W20260709 12:18:18.961122 18369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 852) took 1641 ms (client timeout 1677 ms). Trace:
W20260709 12:18:18.961549 18369 rpcz_store.cc:269] 0709 12:18:17.319477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.319661 (+   184us) service_pool.cc:224] Handling call
0709 12:18:18.960720 (+1641059us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540034ab54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.960946 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407879552}
I20260709 12:18:18.963685 18367 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:18.938427 22503 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.967649 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 751) took 1385 ms. Trace:
I20260709 12:18:18.968032 18294 rpcz_store.cc:276] 0709 12:18:17.582105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.583951 (+  1846us) service_pool.cc:224] Handling call
0709 12:18:17.584370 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:17.584822 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:17.983965 (+399143us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.567843 (+583878us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:18.569230 (+  1387us) log.cc:844] Serialized 52 byte log entry
0709 12:18:18.569489 (+   259us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:18.569852 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.572784 (+  2932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.577421 (+  4637us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.604893 (+ 27472us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18004b4ae0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:18.823765 (+218872us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18004b4ae0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:18.963094 (+139329us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b18004b4ae0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d5d2a6 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1
0709 12:18:18.967297 (+  4203us) raft_consensus.cc:1685] finished
0709 12:18:18.967322 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.967487 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73935,"prepare.queue_time_us":91467,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":105,"raft.queue_time_us":65034,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":369,"spinlock_wait_cycles":109458048,"thread_start_us":828580,"threads_started":2,"wal-append.queue_time_us":612888}
I20260709 12:18:18.968708 22601 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.972704 18444 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:18.973264 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.973568 19037 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:18.974002 18441 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:18.974170 22584 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:18.975617 22234 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.975701 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.976454 18441 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:18.976490 22584 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.977034 22584 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.978107 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.978377 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:18.979001 18444 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:18:18.979755 22190 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.979885 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44108.2us
I20260709 12:18:18.980446 22190 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.981390 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:18.981535 18198 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:18.982190 22415 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.982743 22190 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:18.982774 22415 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:18.983719 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:18.984423 22270 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:18.984685 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68074us
I20260709 12:18:18.984898 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:18.985663 18375 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:18.985755 22378 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:18.986235 22415 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:18.986433 22378 consensus_queue.cc:1243] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:18.987407 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2053) took 1466 ms. Trace:
I20260709 12:18:18.987803 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2062) took 1463 ms. Trace:
I20260709 12:18:18.987792 18367 rpcz_store.cc:276] 0709 12:18:17.521333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.589772 (+ 68439us) service_pool.cc:224] Handling call
0709 12:18:18.963372 (+1373600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.964204 (+   832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.987184 (+ 22980us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540034ab50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:18.987353 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3158030592}
I20260709 12:18:18.988628 22242 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:18.988425 18373 rpcz_store.cc:276] 0709 12:18:17.524113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.636212 (+112099us) service_pool.cc:224] Handling call
0709 12:18:17.636796 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.390013 (+753217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.987532 (+597519us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54000c8fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.987747 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647225600}
I20260709 12:18:18.989490 22530 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:18.989732 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:18.990010 22530 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:18.990388 19036 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:18.989737 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 755) took 1406 ms. Trace:
I20260709 12:18:18.953783 22245 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:18.991529 19042 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:18.991976 22245 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.992413 22245 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:18.967214 22510 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:18.995424 18198 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 5 to 8, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 8 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:18.998675 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.990983 22530 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.999944 22530 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.000437 22378 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.001029 22378 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.983376 22503 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.002687 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:19.003393 19036 raft_consensus.cc:606] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.003760 19036 raft_consensus.cc:614] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:19.004086 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 83045.2us
I20260709 12:18:19.005302 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:19.005792 22595 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:18:19.006019 18449 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:19.008814 22500 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.016996 22503 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.017422 22500 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:19.018816 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.019572 22280 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:19.020776 22500 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:18.989591 22242 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:19.012660 22587 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:19.013669 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.013561 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.015007 22459 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.015516 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.019749 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:18.999044 22490 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:18.991029 18300 rpcz_store.cc:276] 0709 12:18:17.583247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.686740 (+103493us) service_pool.cc:224] Handling call
0709 12:18:17.712040 (+ 25300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.804577 (+1092537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.989491 (+184914us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000393d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:18.989688 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510517120}
I20260709 12:18:19.013564 21648 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:19.034089 21648 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:19.034441 21648 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.035373 21648 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.035611 22251 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.036681 21648 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.036911 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.037580 19036 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:19.037686 22459 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.016203 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.040020 19042 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:18:19.040053 22459 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.041055 22579 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.041566 22579 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.041952 22579 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.043365 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2428) took 1433 ms. Trace:
I20260709 12:18:19.043853 18443 rpcz_store.cc:276] 0709 12:18:17.609708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.192276 (+582568us) service_pool.cc:224] Handling call
0709 12:18:18.192680 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.850510 (+657830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.850558 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.895540 (+ 44982us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.929724 (+ 34184us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:18.929965 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.936887 (+  6922us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.043006 (+106119us) raft_consensus.cc:1685] finished
0709 12:18:19.043029 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.043176 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":629654,"raft.queue_time_us":664111,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"thread_start_us":61166,"threads_started":2,"wal-append.queue_time_us":56973}
I20260709 12:18:19.046289 22445 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.032048 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.050249 18293 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:19.051142 18444 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:19.028909 22587 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.022845 22493 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.051712 22251 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.052307 22587 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.052512 22571 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.053512 22587 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:19.053802 22493 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.055614 22312 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:19.057549 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 221280us
I20260709 12:18:19.057713 22312 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.058557 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:19.059553 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78288us
I20260709 12:18:19.060362 22436 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:19.060855 22436 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.061038 22270 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:19.061303 22436 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.061443 22270 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.061877 22270 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.062155 22415 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:19.063028 22415 consensus_queue.cc:1243] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:18.998363 18198 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 6, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 6 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.064066 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.064927 22442 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.065321 22442 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.065727 22442 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.065536 18198 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:19.066082 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.067622 22470 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:19.067827 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.068823 21635 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.069164 21635 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:19.069478 21635 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.071261 22137 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.026844 18294 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:19.060452 18448 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:18:19.029528 18449 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:19.072573 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.080432 18367 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:19.073675 18374 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:19.012777 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.075552 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.076243 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.076706 22378 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.074239 22242 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.071597 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.081938 19037 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
I20260709 12:18:19.079555 18448 pending_rounds.cc:85] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 7
I20260709 12:18:19.085529 18448 pending_rounds.cc:107] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:18:19.085922 18448 raft_consensus.cc:2889] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.084657 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:19.086843 21635 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.093924 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.094220 19036 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:19.096406 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.087015 19042 raft_consensus.cc:606] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.099444 22459 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.100026 22459 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.103587 22445 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.104564 21635 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.104123 18198 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 8, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 8 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:19.104693 22442 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.064051 22532 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.105822 18198 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 5 to 7, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 7 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:19.106824 22532 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.106997 22245 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.108057 22311 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.108736 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2051) took 2023 ms. Trace:
I20260709 12:18:19.108536 22311 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.109068 18448 rpcz_store.cc:276] 0709 12:18:17.085526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.283620 (+198094us) service_pool.cc:224] Handling call
0709 12:18:17.372650 (+ 89030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.107201 (+1734551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.108521 (+  1320us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400374e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:19.108683 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3880177408}
I20260709 12:18:19.078091 22571 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.048069 22490 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.112396 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.022403 22503 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.106769 22442 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.091717 22251 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:19.110399 22415 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:19.118894 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.128166 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.100503 22459 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.131142 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.134501 22370 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.135488 19035 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:19.135949 18375 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:19.136452 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.138419 22245 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.139313 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.139992 18448 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:19.140668 22270 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.143875 19037 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.126338 22445 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:19.129252 22415 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:19.156505 18444 raft_consensus.cc:2376] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8. Re-sending same reply.
I20260709 12:18:19.164083 19044 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:18:19.166568 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:19.167261 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.167382 18441 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:19.166719 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.146764 22270 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.169400 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.166486 18198 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 8, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 8 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.172227 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:19.168809 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75632.1us
I20260709 12:18:19.176920 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.179755 18449 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8.
I20260709 12:18:19.173071 18448 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:19.149922 22571 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:18:19.201297 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.168689 22415 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:19.169281 22445 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:19.223168 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.225010 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.220137 18198 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.228742 18443 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.234625 18444 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:19.168033 22442 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.241567 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.241904 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.242594 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.238727 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.246131 19035 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.247437 18374 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.173480 22251 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.246837 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.178921 22577 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.252992 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.253010 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.256881 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.258189 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.259007 18293 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:19.259356 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.260810 22311 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:19.262216 22608 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:18:19.262777 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:19.264892 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:19.266031 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:19.271487 22445 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.274752 22370 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.267284 22287 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.276403 22287 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:19.276964 18441 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:19.278448 22287 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.278815 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.280172 18294 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.285691 18443 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:19.257510 19041 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:19.287493 18444 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.287325 19037 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.288940 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.292992 21648 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.294627 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.294806 22579 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.295756 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.296223 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.296514 18293 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:19.295275 19044 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.297360 19036 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:19.297704 22610 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:19.296669 22251 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.300614 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.298291 22234 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:19.255568 22442 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.309187 21648 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.302811 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.304317 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.304695 18440 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.308408 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.305397 18198 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.309630 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.316697 18443 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:19.317368 19043 raft_consensus.cc:2410] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:18:19.316716 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.318158 22442 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.318210 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.318675 18371 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:19.319646 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.320817 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.322273 18293 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:19.323560 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.324296 18297 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:19.325127 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.327838 18443 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:19.328956 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.287557 22503 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.329834 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.330619 18449 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.331800 21648 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.332942 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.333712 18447 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:19.334555 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.339560 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.339735 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.340201 18295 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:19.341017 22370 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.341720 22370 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.343561 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.343900 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.344561 18291 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:19.345871 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.346536 18370 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.347594 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.348265 18298 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:19.348879 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.349471 18297 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:19.346920 22584 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.352110 22584 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:19.354200 22604 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:18:19.355783 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.356581 18295 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:19.359622 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.360220 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:19.361330 22613 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:19.362027 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.362555 18371 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.365171 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.343593 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.367748 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.312752 22283 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.370877 22283 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.373701 22283 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.370827 18371 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.383599 19039 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.384714 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.383843 22137 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.385525 18367 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:19.391889 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.394824 18374 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:19.395769 19035 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.391855 18372 pending_rounds.cc:85] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 9
I20260709 12:18:19.396656 18372 pending_rounds.cc:107] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:18:19.396806 18441 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:19.397120 18372 raft_consensus.cc:2889] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.314599 22579 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:19.401365 18440 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.403015 18444 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.404003 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.407871 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.408926 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.373715 22577 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.414175 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.389873 22190 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:19.415735 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.415783 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.414937 18299 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:19.401023 18198 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:18:19.421748 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.402658 22612 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.422423 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.423612 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.424943 18293 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:19.427516 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.430792 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:19.432129 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.432613 22609 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.002s
I20260709 12:18:19.432967 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.433981 18440 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:19.435541 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.435614 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.436322 18444 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:19.422601 22612 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.441694 18449 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
W20260709 12:18:19.442358 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 203644us
I20260709 12:18:19.443957 19044 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:19.448138 18447 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:18:19.449990 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 142319us
I20260709 12:18:19.400660 22584 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.461266 19037 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:19.450987 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.462992 18446 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.468394 18294 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:19.473039 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.469969 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.474846 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.479305 19044 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:19.431066 22415 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.481839 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:19.489287 18296 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.483340 18367 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:19.436352 22190 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:19.489197 18291 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:19.451448 22612 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.498095 18198 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 8, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 8 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.499032 18449 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.503439 18366 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:19.505520 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.508183 19035 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:19.509467 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.510848 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.512454 18295 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:19.514374 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.488814 22245 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.521927 18198 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 7, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 7 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:19.523772 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.525087 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.526300 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.527549 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.528066 18449 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.528676 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.530239 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.538813 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.539454 22503 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:19.540158 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.540889 18447 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:19.545980 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.526899 18443 raft_consensus.cc:2376] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11. Re-sending same reply.
I20260709 12:18:19.552325 18370 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:19.552803 18449 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
W20260709 12:18:19.556181 22607 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.310s	user 0.003s	sys 0.000s
I20260709 12:18:19.542460 22503 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:19.557634 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.571259 18368 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:19.558337 18198 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 6, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 6 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:19.557284 18440 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:19.560808 18449 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.560832 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.571441 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.572719 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:19.572932 18370 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:19.563634 22490 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.577409 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.579118 18442 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:19.582470 18296 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:19.587539 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.587725 18198 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:19.589903 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.590485 18374 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.591199 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.581655 18367 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:18:19.578867 22587 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.000s
I20260709 12:18:19.567924 22137 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:19.596935 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:19.597288 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.597973 18441 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.593365 22370 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.601220 22370 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:19.602474 18338 leader_election.cc:400] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8.
I20260709 12:18:19.602808 18338 leader_election.cc:403] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:19.607429 22530 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8.
I20260709 12:18:19.575625 22490 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:19.608486 22490 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.608294 22530 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:19.611634 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.612389 18366 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.590668 22584 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:19.598763 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.622388 22615 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:18:19.623103 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.624766 22587 raft_consensus.cc:993] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:19.594937 22378 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:19.568727 22436 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.633122 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.633749 18369 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.637190 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:19.597023 22137 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.639719 22137 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.640431 18373 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:19.648307 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.649869 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:19.651832 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69411.2us
I20260709 12:18:19.629081 22579 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:19.594307 22245 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.689494 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.697201 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:19.655220 18446 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:19.657132 19037 raft_consensus.cc:1217] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[8.7-9.8]   Dedup: 8.7->[9.8-9.8]
I20260709 12:18:19.700050 19037 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: [8.7-9.8]
I20260709 12:18:19.630201 22584 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:19.649139 19039 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:19.706634 18441 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:19.710695 18366 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:19.660490 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2430) took 2045 ms. Trace:
I20260709 12:18:19.661152 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.663439 22503 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:19.667384 22500 raft_consensus.cc:1081] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:19.711467 18445 rpcz_store.cc:276] 0709 12:18:17.610169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.222450 (+612281us) service_pool.cc:224] Handling call
0709 12:18:18.222780 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.936363 (+713583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.939252 (+  2889us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.031074 (+ 91822us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.098739 (+ 67665us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:19.098784 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.098897 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.482348 (+383451us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:19.643384 (+161036us) raft_consensus.cc:1685] finished
0709 12:18:19.643414 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.655871 (+ 12457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":599952,"raft.queue_time_us":634225,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"spinlock_wait_cycles":6735232,"thread_start_us":119464,"threads_started":2,"wal-append.queue_time_us":123593}
I20260709 12:18:19.670100 22234 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:19.673362 22137 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:19.713441 18448 raft_consensus.cc:2368] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12.
I20260709 12:18:19.704177 18374 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:18:19.717154 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 84119.9us
I20260709 12:18:19.651587 22577 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:19.720445 18370 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:19.727101 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.727869 19044 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.746791 21648 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:19.747365 21648 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.715639 22584 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.749168 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:19.749341 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52260us
W20260709 12:18:19.726866 22459 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.569s	user 0.006s	sys 0.005s
I20260709 12:18:19.721459 22615 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:19.771965 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:19.743923 22500 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:19.773694 22500 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.707320 22378 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:19.721323 22577 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:19.719363 22579 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.757903 21648 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.708890 22612 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.750556 22415 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:19.755071 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65670.8us
I20260709 12:18:19.782208 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:19.782552 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:19.782974 19037 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.778120 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:19.787832 18440 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.789604 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:19.792625 19038 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:19.754297 22510 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:19.795703 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 361349us
I20260709 12:18:19.791603 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.794618 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.791327 22615 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:19.775528 22378 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:19.804329 19035 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.781416 22577 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.774817 22500 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.780295 22459 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.797386 22510 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.817094 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.821270 21648 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.808137 22532 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:19.828378 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:19.842563 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48057.5us
I20260709 12:18:19.840258 19042 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:19.835841 19044 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:19.822432 22510 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:19.834612 22503 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:19.849484 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:19.846405 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.850806 22234 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:19.850962 22242 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:19.835021 22500 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.837874 22251 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:19.860544 22490 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:19.851712 22618 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:18:19.868378 22571 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.869058 22490 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.871383 22532 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.854367 22579 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:19.877933 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:19.886294 19037 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:19.853102 22503 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.863420 22234 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:19.865269 22436 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:19.882995 22571 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.907680 22503 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:19.915977 22245 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:19.928565 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50739.1us
I20260709 12:18:19.929378 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.884692 22532 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.909670 22137 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.935964 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:19.940546 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 336552us
I20260709 12:18:19.940858 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.965205 18374 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.964851 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.970001 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:19.972317 18373 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:19.973254 18375 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:19.942414 18296 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:19.975761 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:19.976493 18366 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:19.978145 18294 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:19.933295 22532 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.932706 22571 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.938612 22436 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.004299 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:20.004871 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.003451 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.980758 18299 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:20.011174 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:20.011673 18293 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:20.012161 18300 raft_consensus.cc:2368] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:19.933382 22311 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.002962 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:19.988305 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:20.015421 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49019.4us
I20260709 12:18:19.884347 22378 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.030371 18366 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:18:20.032473 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44273.1us
I20260709 12:18:20.008464 22251 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:20.008128 22283 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:20.033406 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.043347 22234 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.009469 22436 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:20.047497 22459 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.047856 22378 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:20.048849 22619 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:18:20.049211 22490 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:20.049999 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.018033 22452 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:20.077907 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44597.9us
I20260709 12:18:20.022756 22612 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:19.964841 22190 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.083810 22452 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.060570 22436 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.088981 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:20.061169 22415 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.060572 22459 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:20.087703 22283 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:20.106405 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.086164 22251 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.092247 22452 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:20.090070 22490 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.118969 18369 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:20.117203 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 301078us
W20260709 12:18:20.121551 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 293242us
I20260709 12:18:20.121521 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.128798 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:20.129500 18300 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.130272 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 290316us
W20260709 12:18:20.131057 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 341540us
I20260709 12:18:20.105829 22190 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.108088 22615 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:20.152431 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.110400 22625 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.158192 18298 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.102525 22280 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:20.164539 18292 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:20.147888 22370 raft_consensus.cc:2764] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:20.166051 22251 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.168833 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:20.175613 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:20.175634 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:20.177094 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:20.178766 22283 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.180945 22627 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.183815 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:20.186064 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64674.1us
I20260709 12:18:20.188426 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:20.189749 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 260461us
W20260709 12:18:20.196031 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 178244us
I20260709 12:18:20.167606 22415 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.220781 19044 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:18:20.221207 19044 pending_rounds.cc:85] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 10
I20260709 12:18:20.223029 19044 pending_rounds.cc:107] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:18:20.223331 19044 raft_consensus.cc:2889] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.223598 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:20.224555 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:20.227895 19035 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.234743 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.185194 22459 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.241569 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.165109 22625 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.240180 19042 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:20.199504 22627 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:20.231021 22370 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.252291 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.213328 22415 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:20.257339 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.258386 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.232816 22280 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:20.255363 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.257331 19038 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:20.260315 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.192023 22510 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.262410 18447 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:20.264690 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49729us
I20260709 12:18:20.246148 22636 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.268601 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:20.269924 18445 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:20.317180 18300 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:20.257516 22370 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:20.264320 22415 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:20.335008 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 86129.6us
I20260709 12:18:20.337687 19043 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:20.340698 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 73189.8us
I20260709 12:18:20.249943 22459 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:20.316601 22620 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:18:20.344066 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 167067us
I20260709 12:18:20.238968 22490 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:20.342732 22577 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.001s
I20260709 12:18:20.341039 22370 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.364003 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 106722us
I20260709 12:18:20.364742 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:20.370096 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 186334us
I20260709 12:18:20.370481 21648 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.366866 22311 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.333575 22639 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.368000 18371 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:20.348994 22280 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.371058 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.370954 21648 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:20.402742 21648 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.399868 19037 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:20.256811 22627 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.346733 22637 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.262225 22615 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.406965 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.411270 18372 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.330231 22641 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 12:18:20.388659 18375 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:18:20.374749 22251 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.371951 22311 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:20.388147 22639 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.325047 22636 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.425057 21648 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.404832 22280 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.428432 22311 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.409714 22283 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:20.357302 22640 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 12:18:20.431809 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80619.5us
I20260709 12:18:20.373390 22577 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:20.444772 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 95872.5us
I20260709 12:18:20.449047 19038 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:20.449542 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.449687 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.454437 21648 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:20.455224 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 216373us
I20260709 12:18:20.455885 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.405057 22503 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.457382 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:20.434612 22639 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.459380 22283 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.461524 22283 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:20.463088 22577 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.442727 22280 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.414211 22637 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.464186 22459 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.465636 22637 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:20.466249 22584 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.004s
I20260709 12:18:20.468289 22627 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.469436 22640 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:20.471828 22625 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.466661 22584 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.479729 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.482453 18369 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:20.482491 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.496395 19042 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:20.444406 22500 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:18:20.500458 18264 leader_election.cc:400] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12.
I20260709 12:18:20.501068 18264 leader_election.cc:403] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:20.503521 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.504354 18374 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:20.483489 22577 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.510644 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.515426 18369 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:20.516782 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.471144 22532 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.483330 22625 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:18:20.524289 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 162091us
I20260709 12:18:20.527141 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.529048 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.530683 22645 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.531224 22645 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.530690 22584 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.532060 22234 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:20.532208 22645 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:20.536248 22648 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:20.517418 22577 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.541631 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.522567 22459 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.519058 22510 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.525647 22599 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.525885 22251 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:20.549520 18443 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:20.553777 22242 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.566431 22242 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.567193 22242 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.568360 22242 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.529177 22656 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:20.546633 22648 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:20.518795 22436 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.558089 22280 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.540683 22234 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:20.549239 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:20.545418 22577 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.554750 22642 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:20.556175 22622 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.522s	user 0.002s	sys 0.002s
I20260709 12:18:20.558398 22661 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:20.559196 22470 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:20.559957 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:20.562582 22503 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.562420 22311 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.547729 22459 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.519110 22659 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:18:20.601315 22459 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:20.601670 22571 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 12:18:20.563766 22283 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:20.602085 22571 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.571643 22656 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.580391 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.571864 22636 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.580825 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.581192 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.605605 22656 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.554139 22599 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:20.606911 22599 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.571205 22370 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:20.581763 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:20.611172 22650 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:18:20.602898 22311 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.611879 22251 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:20.583974 22624 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.475s	user 0.002s	sys 0.000s
I20260709 12:18:20.612845 22251 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.615720 22599 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.609354 22370 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:20.616757 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 67595.6us
I20260709 12:18:20.617413 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:20.591032 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.598773 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:20.617648 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:20.623191 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.574344 18298 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:20.639379 18366 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:20.643057 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:20.589012 18372 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:20.651252 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.651147 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:20.699612 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.701331 19042 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:20.700083 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.652374 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:20.703502 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.704818 18442 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:20.589260 22646 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.002s
I20260709 12:18:20.652648 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.602751 22283 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.584657 22436 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:20.717628 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 268119us
I20260709 12:18:20.709874 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:20.705123 18449 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.719033 19043 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:20.715739 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.639333 22245 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.586107 22137 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:20.704604 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64367.4us
I20260709 12:18:20.717113 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.721596 18370 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14.
I20260709 12:18:20.612613 22452 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.597366 22470 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:20.586615 22651 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.726306 18375 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:20.726697 18375 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.588624 22280 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:20.589080 22532 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.595413 22642 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.578218 22664 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.731539 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:20.630892 22625 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.631060 22190 raft_consensus.cc:2764] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:20.599148 22667 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:20.706182 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 107465us
I20260709 12:18:20.740500 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:20.724457 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.611388 22639 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.720167 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:20.736395 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 365384us
I20260709 12:18:20.723001 22137 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.738933 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.716125 22283 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.736466 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:20.641438 22661 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:18:20.749244 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:20.743691 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:20.759099 18440 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:20.759617 18440 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:20.760485 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.735334 22664 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:20.759949 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:20.764362 18186 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.402s	user 0.000s	sys 0.000s
I20260709 12:18:20.764173 22388 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:20.775830 22388 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.775655 22655 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:20.776224 22388 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:20.776453 22655 consensus_queue.cc:1243] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:20.777122 22388 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.778288 22388 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.785208 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:20.727964 22245 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.743805 22637 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:20.789091 18370 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:20.779385 22442 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:20.742900 22436 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:20.587973 22234 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.788787 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:20.746685 22190 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.805696 22190 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.806207 22190 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:20.806696 22190 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.748378 22311 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:20.797070 18447 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:20.750120 22645 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:20.799441 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67985us
I20260709 12:18:20.744392 22625 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:20.799548 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.791199 18440 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.786394 22532 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.787016 22640 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:20.799762 22370 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:20.793174 22667 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:20.811349 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93328.3us
I20260709 12:18:20.826687 18440 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.833431 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.808979 22659 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:20.837983 18366 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:18:20.838081 18442 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:20.838634 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.843029 18449 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:20.786427 22577 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.840444 18370 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.851081 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.803076 22503 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:18:20.823925 22657 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.854646 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:20.834652 22645 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.866986 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.837740 22370 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.860998 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:20.810241 22637 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:20.867638 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:20.854375 22667 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.810778 22436 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.862707 22648 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12.
I20260709 12:18:20.856886 22659 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.869841 22452 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:20.870685 22599 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:20.813253 22470 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:20.913221 22670 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:18:20.913667 22669 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:18:20.914040 22667 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.916540 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.916960 22530 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:20.830489 22137 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:20.944979 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 184353us
I20260709 12:18:20.948103 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.953994 22626 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.898653 18440 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:18:20.930214 18371 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:20.930928 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 63395us
I20260709 12:18:20.960434 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.961457 18440 pending_rounds.cc:85] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 9
I20260709 12:18:20.962366 18440 pending_rounds.cc:107] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:18:20.847533 22283 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.962620 18440 raft_consensus.cc:2889] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.976519 18444 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:20.977036 18440 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:20.979467 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.980614 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:20.849725 22642 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.987489 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:20.988312 18371 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:20.989801 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:20.991484 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:20.992100 18443 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:18:20.978600 22490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.025s
I20260709 12:18:20.992834 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:20.981787 22655 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:18:20.949980 22530 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.929030 22625 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:20.977952 22648 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:20.977034 22280 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:20.879289 22664 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:20.985467 22640 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.038745 21648 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:20.981796 22442 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.984803 22626 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:21.011037 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.029960 19042 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.977422 22645 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:20.982345 22370 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.133819 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.067898 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.136411 18373 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 6.
I20260709 12:18:21.035674 22245 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:21.130414 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 137649us
W20260709 12:18:21.031409 22682 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:18:20.995865 22661 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:20.999788 22490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.026s
I20260709 12:18:21.077148 18375 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:21.031886 22676 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 12:18:21.128822 22459 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
I20260709 12:18:21.161139 22459 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.138777 18186 maintenance_manager.cc:643] P 79114be45f9144e78102be8765ae6af5: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.728s	user 0.157s	sys 0.076s Metrics: {"bytes_written":29217,"cfile_init":1,"dirs.queue_time_us":123473,"dirs.run_cpu_time_us":599,"dirs.run_wall_time_us":37443,"drs_written":1,"lbm_read_time_us":378,"lbm_reads_lt_1ms":4,"lbm_write_time_us":278127,"lbm_writes_10-100_ms":8,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":63,"thread_start_us":35265,"threads_started":1}
I20260709 12:18:21.133193 22442 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.164269 22459 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:21.165246 22640 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.165827 22442 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:21.061390 22680 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:18:21.123091 22577 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:21.147439 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.997777 22283 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.170580 18187 maintenance_manager.cc:419] P 79114be45f9144e78102be8765ae6af5: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10302 bytes on disk
I20260709 12:18:21.171407 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.171572 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.172176 18367 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:21.172521 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.172178 18369 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.163663 22530 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.175005 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.175631 18367 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:21.176725 18186 maintenance_manager.cc:643] P 79114be45f9144e78102be8765ae6af5: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":296,"lbm_reads_lt_1ms":4}
W20260709 12:18:21.153157 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.178081 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.178836 18374 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:21.017557 22639 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:20.996649 22311 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.030828 22436 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:21.180692 22415 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.833s	user 0.011s	sys 0.000s
I20260709 12:18:21.132911 22532 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.181181 22415 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.181633 22532 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.158070 18373 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:18:21.183400 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.183662 22651 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.183938 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:20.994372 22637 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.185233 18373 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:21.019937 22685 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.133670 22242 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:21.185113 22415 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.189157 22242 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:21.185118 22459 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:18:21.190107 22242 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.176962 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.184562 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.030260 22672 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.030441 22643 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 12:18:21.163645 19043 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:21.193603 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.203244 18367 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:21.184096 22651 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:21.190523 22668 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:21.211028 18368 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:21.211529 18368 pending_rounds.cc:85] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:18:21.211800 18368 pending_rounds.cc:107] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:21.212075 18368 raft_consensus.cc:2889] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:21.179651 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.181532 22311 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.137286 22626 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:20.999881 22659 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.207832 22668 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.268203 18367 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:21.155725 22661 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:21.156132 22673 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:18:21.204820 22664 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.236646 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.274875 18374 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:21.255865 18369 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:21.142765 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:21.258394 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:21.276443 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 133776us
I20260709 12:18:21.277562 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:21.278627 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:21.279156 22652 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:21.279371 22648 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:21.279731 22652 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:21.280045 22648 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:21.280694 22652 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.280865 22648 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:21.282450 18415 leader_election.cc:336] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.512s (SENT)
I20260709 12:18:21.278724 22242 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:21.286599 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.257611 22378 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:21.286832 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:21.288094 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:21.289880 22639 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:21.290207 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:21.292270 18412 leader_election.cc:400] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:21.292644 18412 leader_election.cc:403] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:21.295356 22672 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:21.295529 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.295926 22672 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.298080 22672 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.298722 22452 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:21.299247 22452 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:21.300165 21648 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:21.300086 22452 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.300643 21648 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:21.301718 22672 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.302250 22672 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.303339 22442 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:21.303715 22442 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.304042 22442 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.304888 22672 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.305914 22490 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.306388 22490 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.307304 22436 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:21.307687 22679 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:21.308243 22242 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.308411 22490 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.308753 22242 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.309783 18367 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:18:21.310930 22683 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 12:18:21.308251 22679 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.311332 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.310837 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.310830 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.316155 21648 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.317164 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.320740 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.321642 21648 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:21.322095 18368 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:21.327533 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.327764 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:21.328258 18374 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:21.328514 18449 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:21.311371 22459 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.331451 22388 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:18:21.332083 18368 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:21.334939 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.335484 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.392342 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.335723 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.322222 22510 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:18:21.318336 22684 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:21.393965 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.316753 22690 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:21.337404 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.396042 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.397500 18449 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:21.338577 22626 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:21.344516 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.344981 18369 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:21.289327 22378 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:21.400434 22378 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.401625 22378 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.345774 18375 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.404556 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56140 (request call id 877) took 1038 ms. Trace:
I20260709 12:18:21.404950 18375 rpcz_store.cc:276] 0709 12:18:20.366062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.388504 (+ 22442us) service_pool.cc:224] Handling call
0709 12:18:21.342116 (+953612us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54000c85d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e73a3d 00007f3c6de43cd0 00007f3c673c8254 00007f3c673c81e2 00007f3c673c8111 00007f3c673c808c 00007f3c673c7342 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:21.404437 (+ 62321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702180096}
I20260709 12:18:21.316452 22242 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:21.407622 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.343703 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.389245 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.398171 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.410271 18297 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 7.
I20260709 12:18:21.399106 18298 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 12:18:21.357218 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 366994us
I20260709 12:18:21.412195 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.412623 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.412931 18291 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:21.412818 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:21.413555 22686 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:18:21.402697 22685 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:18:21.413820 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.413880 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.415015 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.413928 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.415932 18295 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.416337 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:21.417218 22683 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:21.417739 18298 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:18:21.418066 18298 pending_rounds.cc:85] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 8
I20260709 12:18:21.418293 18298 pending_rounds.cc:107] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:18:21.418520 18298 raft_consensus.cc:2889] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:21.420645 18371 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:21.426476 18291 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:21.431476 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.432302 18294 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:21.347162 22137 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.434890 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.436731 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.437107 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.437719 18375 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.320492 22656 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:18:21.343159 22661 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.450730 22645 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:21.451481 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:21.451619 22645 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:21.452235 22645 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:21.453190 22645 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.357867 22532 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:18:21.469548 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.490041 18291 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:21.493584 18415 leader_election.cc:336] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.726s (SENT)
I20260709 12:18:21.494110 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:21.495162 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:21.495846 18297 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:21.501662 18371 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:21.411392 22459 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.503188 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:21.474110 22687 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
W20260709 12:18:21.504706 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 180500us
I20260709 12:18:21.505015 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.508608 18371 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14.
I20260709 12:18:21.508534 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:21.512161 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.515359 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:21.488792 22672 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:21.516412 22532 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:21.517171 22532 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:21.517495 22532 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.518246 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:21.518313 22532 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.519210 22388 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:21.519498 22639 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:21.520414 22639 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.520736 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:21.521574 22639 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.521872 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:21.522500 22388 raft_consensus.cc:2764] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:21.523222 22532 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.529812 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:21.531119 18295 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:21.542766 19038 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:18:21.494901 22645 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.516371 22672 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.532797 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.549805 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:21.554118 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:21.555173 18375 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
W20260709 12:18:21.558136 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64579.8us
W20260709 12:18:21.559561 18415 leader_election.cc:336] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.764s (SENT)
I20260709 12:18:21.560130 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:21.553465 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:21.565865 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.566015 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:21.566435 18375 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:21.535337 22685 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:21.535318 22436 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:21.564544 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:21.578228 18447 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.426664 22668 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.551616 22452 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.554801 22626 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.561267 22577 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.557729 22490 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.532161 22673 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:21.569392 22445 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:21.588371 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.580806 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:21.595605 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.603951 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.608047 18371 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: [8.8-12.10]
I20260709 12:18:21.614678 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.615360 18368 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:21.617084 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.618177 18369 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:18:21.619791 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 304851us
W20260709 12:18:21.621829 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:21.622391 18415 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.854s (SENT)
I20260709 12:18:21.622861 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:21.622519 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.627302 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:21.629032 22665 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:21.629547 22665 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.630012 22665 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:21.642576 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.610323 22679 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.651153 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:21.637413 22657 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.759s	user 0.007s	sys 0.008s
I20260709 12:18:21.665012 18375 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:21.657271 18369 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:21.614530 22668 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:21.579068 22651 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:21.667084 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.627331 22656 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.613297 22445 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:21.595175 22626 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.589556 22452 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.648977 22436 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.656888 22510 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:21.712736 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85522.7us
I20260709 12:18:21.636391 22415 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:18:21.636956 22283 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:21.669808 22657 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.667575 22681 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.719122 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.702157 22667 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.721238 18442 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:18:21.792555 18442 pending_rounds.cc:85] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:21.792884 18442 pending_rounds.cc:107] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:21.793186 18442 raft_consensus.cc:2889] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:21.793286 18297 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:21.725195 18369 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:21.781795 18366 raft_consensus.cc:2368] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12.
I20260709 12:18:21.719552 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.791635 18292 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
W20260709 12:18:21.828920 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 206537us
W20260709 12:18:21.831992 18415 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.867s (SENT)
I20260709 12:18:21.840377 18296 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:21.842483 18375 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:21.843253 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.844748 18369 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.844610 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:21.851990 22706 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.852627 22672 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.854732 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:21.858800 22577 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.862849 18294 raft_consensus.cc:2410] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:21.795516 22660 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.863386 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:21.863564 22660 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.864212 18375 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:21.865775 22660 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:21.867412 22445 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.869796 18372 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:21.732174 22510 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.881384 22510 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:21.882783 22190 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.075s	user 0.010s	sys 0.002s
I20260709 12:18:21.883065 18369 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:21.888567 22452 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:21.727171 22644 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.278s
I20260709 12:18:21.895547 22644 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:18:21.897696 22709 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:18:21.713521 22656 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:21.913035 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:21.913820 22700 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:18:21.914053 18369 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:21.915354 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 83391.7us
I20260709 12:18:21.916249 22640 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:21.916422 22704 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:18:21.916671 22640 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:21.917938 22712 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260709 12:18:21.917536 22640 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:21.918548 22640 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.919571 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:21.920253 18440 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:21.610435 22639 consensus_queue.cc:1175] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" responder_term: 14 status { last_received { term: 14 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:21.921280 22532 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.922189 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:21.833935 22459 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:21.916251 18415 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.876s (SENT)
I20260709 12:18:21.935814 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:21.934356 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:21.844592 22652 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:21.905834 22696 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:18:21.780980 22442 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:21.936730 22378 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.001s
I20260709 12:18:21.832152 22667 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:21.966956 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50750.1us
I20260709 12:18:21.967589 22716 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:21.971709 22714 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:18:21.992645 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.943322 22652 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.011595 22656 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:21.990011 22442 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:21.993063 22655 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.013098 22655 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.004801 22721 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.015852 22721 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.018513 22721 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:21.883327 22190 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.021173 22190 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.022001 18298 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:22.014063 22667 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.025874 22655 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:21.990420 22626 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.014573 22652 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.029634 22652 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:22.034410 18371 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:22.040936 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.041433 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.041805 18368 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:22.042519 18369 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:22.043455 18415 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 2.109s (SENT)
I20260709 12:18:22.043996 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.045621 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.046504 18375 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:22.047389 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.047590 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.048610 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.048939 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.049755 22652 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.049975 18369 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:22.050217 22655 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:22.050356 22652 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.051272 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.052292 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.052580 22652 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.052889 18368 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:22.050993 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.054490 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.055466 18369 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:22.049829 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.055290 18368 raft_consensus.cc:2376] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
I20260709 12:18:22.060289 19042 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:22.061578 18447 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:22.086283 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.088694 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:22.091619 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:22.092069 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.092865 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.093760 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:22.103138 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:22.106287 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.111865 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1004) took 1256 ms. Trace:
I20260709 12:18:22.126303 19043 rpcz_store.cc:276] 0709 12:18:20.849121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.849330 (+   209us) service_pool.cc:224] Handling call
0709 12:18:20.870541 (+ 21211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.202809 (+332268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.105214 (+902405us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400747f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:22.105419 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673412992}
I20260709 12:18:22.093036 22137 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:22.097199 22577 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:22.107285 22625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.016s	sys 0.008s
W20260709 12:18:22.134661 22625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.016s	sys 0.008s
I20260709 12:18:22.134456 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.138615 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:22.141208 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.139645 22459 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.135401 22577 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.146458 18440 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:22.142638 22612 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.152784 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.153264 18375 raft_consensus.cc:2376] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8. Re-sending same reply.
W20260709 12:18:22.148877 22661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.018s
I20260709 12:18:22.137658 22137 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:22.154161 22661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.018s
I20260709 12:18:22.147568 22577 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.165323 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:22.169621 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.174216 18295 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:18:22.167826 22665 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.007s
W20260709 12:18:22.178349 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:22.178875 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.161967 22723 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:22.146507 22459 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.167523 22724 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:22.185885 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.186491 18375 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:22.188251 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.188992 18296 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:22.190383 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.190435 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.191253 18293 raft_consensus.cc:2368] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9.
I20260709 12:18:22.191570 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.192298 18293 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [10.8-12.9]
I20260709 12:18:22.192613 22571 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.193303 18292 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:22.194401 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.196132 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.196213 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.193461 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.196947 19042 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.199697 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.164669 22711 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:18:22.201617 22651 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.201887 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.203179 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.203773 22137 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.206296 22459 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:22.206902 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2099) took 1380 ms. Trace:
I20260709 12:18:22.155454 22612 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.196867 19044 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:22.208289 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.208833 19037 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:22.209115 19037 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:22.191622 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.207334 18448 rpcz_store.cc:276] 0709 12:18:20.826665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.826814 (+   149us) service_pool.cc:224] Handling call
0709 12:18:20.827108 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.930957 (+103849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.930998 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:20.931039 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:20.931076 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.206691 (+1275615us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54003b24d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:22.206735 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.206851 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238370560}
I20260709 12:18:22.215862 18291 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:22.219852 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.220871 18292 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:22.209971 19037 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.221804 19037 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.220511 19043 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:22.224978 19039 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEARNER]: Advancing to term 10
I20260709 12:18:22.225284 18297 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:22.225504 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:22.226260 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:22.227038 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.228183 22684 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.228634 22684 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.229120 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.230201 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.230623 22684 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.230720 18442 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:22.231379 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.232110 18293 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:22.232443 19042 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:22.233752 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.234339 18440 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:18:22.235674 22731 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:18:22.240058 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.247292 18442 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:22.254720 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.179735 22665 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.236893 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.255791 18374 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:22.260918 18369 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:22.255786 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:22.263008 19037 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:22.262284 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:22.264298 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:22.234118 22723 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:22.264358 22661 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:18:22.266023 22729 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:22.247704 22626 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:22.239959 22724 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:22.266510 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:22.269289 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 136768us
I20260709 12:18:22.266809 18375 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:22.271605 18291 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:22.272389 18374 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: [8.9-13.11]
I20260709 12:18:22.275004 18374 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:22.276822 22445 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:22.281154 22445 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:22.281486 22445 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:22.281697 18368 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:22.282332 22445 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.282683 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.263181 22571 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:22.268505 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7887573059999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 12:18:22.268796 22722 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 12:18:22.283581 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7707573059999504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
I20260709 12:18:22.283912 22445 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.279960 18291 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:18:22.284053 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7239573059998747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:18:22.284732 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:22.285735 22612 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:22.285887 22732 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:22.285902 22283 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:22.286516 22732 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:22.286892 22732 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:22.286926 22719 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:22.287207 22661 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:22.287988 22648 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:22.287884 22625 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:22.288455 22648 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.288583 22625 consensus_queue.cc:1243] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:22.267544 22711 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:22.289307 22648 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.289932 22711 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.290083 22726 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.290366 22711 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.290508 22648 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.284735 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.278985 22640 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:22.291980 22626 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.292603 18375 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:22.293370 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.293907 18375 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:22.294099 22626 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:22.285096 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7239573059998747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 12:18:22.295145 18338 connection.cc:256] RPC call timeout handler was delayed by -1.624957305999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 12:18:22.264767 22662 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.295537 18338 connection.cc:256] RPC call timeout handler was delayed by -1.5817573059998722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
I20260709 12:18:22.295785 22662 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.295267 18375 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:22.291566 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.310734 18441 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:22.302476 22662 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.315156 18441 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:22.283840 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.305698 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.317801 18442 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:22.300736 19039 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:22.318775 18374 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:18:22.318980 18338 connection.cc:256] RPC call timeout handler was delayed by -1.5745573059998605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
I20260709 12:18:22.319641 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:22.320132 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1578573059998689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 12:18:22.317721 18368 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:22.324440 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:22.315884 22724 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.332644 22640 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.334053 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.334940 22640 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.335523 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.336261 22234 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:22.336609 22640 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.336802 22234 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.337273 22234 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.337227 22640 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:22.337963 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.338748 22640 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.339387 22612 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:22.311381 22723 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.339442 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.339484 22684 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.269785 22729 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.290634 22726 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.287938 22661 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:22.327275 22510 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.346750 22729 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.330384 22639 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:22.350747 22684 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.373334 22459 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.374768 22459 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.376976 22459 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:22.376725 22711 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.379472 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.357854 22665 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.378934 22711 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.351950 22510 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.385833 22510 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.381551 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.386698 22668 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.389393 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.390255 22668 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.397157 22470 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.398145 22668 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.399565 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.400872 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.401641 22655 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.402298 22655 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.403390 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.404101 22668 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.382887 22716 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:22.421399 18370 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1887) took 1555 ms (client timeout 1876 ms). Trace:
W20260709 12:18:22.421852 18370 rpcz_store.cc:269] 0709 12:18:20.866340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.866695 (+   355us) service_pool.cc:224] Handling call
0709 12:18:22.421189 (+1554494us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54004150d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:22.421340 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2923045248}
I20260709 12:18:22.428613 19038 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
W20260709 12:18:22.433261 22738 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:22.435144 22726 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.437635 22577 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.374100 22639 consensus_queue.cc:1243] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:22.441193 18441 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:22.441607 22729 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.423141 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:22.441627 18441 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:22.445011 18441 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.445751 22234 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.446453 18441 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.447481 22732 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.447912 22234 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.449496 22732 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.454036 18447 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:22.456290 22577 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.457625 22729 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.459776 22242 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:18:22.466158 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.473807 22577 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:22.487357 22684 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.508725 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:22.509709 22727 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:22.510241 22727 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.511062 22727 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.513371 22727 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:22.537143 22242 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:22.538295 22242 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:22.538703 22242 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.541180 22242 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:22.541796 19006 connection.cc:256] RPC call timeout handler was delayed by -1.0360761029999139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:18:22.542099 22470 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.545866 22599 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 12:18:22.550792 22713 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.667s	user 0.000s	sys 0.008s
I20260709 12:18:22.551000 22242 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.553675 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:22.567205 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.569877 18446 pending_rounds.cc:85] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:22.570832 18448 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:22.571336 18446 pending_rounds.cc:107] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:22.571712 18446 raft_consensus.cc:2889] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:22.573624 22652 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:22.575371 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.577492 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:22.578022 18441 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.578411 18448 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:22.582001 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.582572 18444 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:22.566259 22470 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.587450 22470 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.598867 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.598924 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.599833 18448 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:18:22.610327 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 56724.1us
I20260709 12:18:22.614064 18441 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:22.612455 22470 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.624595 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.625600 22470 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:22.626047 22470 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:22.626492 22470 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:22.626647 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.628810 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.629628 22729 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.630801 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.631606 22729 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:22.632899 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.622218 22640 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:22.633674 22729 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.633843 22640 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:22.634181 22470 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.634955 22640 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.636876 22729 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:22.640451 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.643527 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:22.642827 22724 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.650527 22660 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:22.653183 22660 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:22.654678 22251 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.655180 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.655193 22251 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.657557 22251 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.657627 22648 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.658073 22251 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.658154 22648 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.658561 22251 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.659782 18292 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:22.652571 19043 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:22.660724 22732 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.654369 22660 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.660749 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.662259 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.662809 18295 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.663122 22470 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:22.665179 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.666242 18292 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:22.666939 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.667629 19039 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.661211 22732 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.660334 22648 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.669245 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.669802 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.670082 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.677644 18292 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:22.679039 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.670732 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.671692 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1953) took 1270 ms. Trace:
I20260709 12:18:22.682021 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.682778 18292 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:22.683952 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.687215 19039 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:22.680428 18298 rpcz_store.cc:276] 0709 12:18:21.401266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.431045 (+ 29779us) service_pool.cc:224] Handling call
0709 12:18:21.431449 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.022487 (+591038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.671480 (+648993us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54002760d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:22.671637 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357670784}
I20260709 12:18:22.706130 19039 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:22.706584 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.707597 18441 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:22.709827 18295 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:22.710597 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.707618 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.712738 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.713337 18298 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:22.713618 19039 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:22.714475 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.715847 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.716177 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.716602 19035 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:22.718693 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.720571 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.721084 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.721940 18298 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:22.722381 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.722894 18295 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:22.723579 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:22.724502 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.726217 18295 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:22.728168 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.715540 22242 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.735558 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.735785 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.736158 19044 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:22.736563 22732 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:22.719038 22685 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:18:22.737097 22732 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:22.737336 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.737934 22685 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.738263 22732 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:22.738412 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.738569 22685 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.739480 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.739583 22251 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:22.739902 22727 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:22.740028 19035 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.740492 22727 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.740738 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:22.741400 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:22.742624 22661 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:22.744410 18298 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:22.744663 22661 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:22.745906 22661 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.746596 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:22.747272 18440 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:22.742846 22571 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:18:22.748236 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:22.748765 18298 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:22.741420 22727 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.746843 22661 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.752034 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:22.731129 18292 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:22.752842 19035 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:22.754648 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.752733 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:22.675364 22684 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.761270 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:22.762816 18292 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:18:22.763353 18292 pending_rounds.cc:85] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 7
I20260709 12:18:22.763608 18292 pending_rounds.cc:107] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:18:22.763986 18292 raft_consensus.cc:2889] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:22.763571 22684 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:22.767328 19039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:22.769096 18292 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:22.770336 19039 raft_consensus.cc:606] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.770686 19039 raft_consensus.cc:614] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:22.738765 22732 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:22.774514 18292 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:22.783588 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:22.785630 22445 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:22.788715 22707 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.060s	user 0.003s	sys 0.000s
I20260709 12:18:22.784801 19039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:22.792038 19039 raft_consensus.cc:606] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:22.764515 22251 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:22.761978 22490 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:22.966500 22732 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.443197 22732 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:22.797799 22251 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.447535 22251 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:23.096467 22718 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.299s	user 0.003s	sys 0.000s
W20260709 12:18:22.806149 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.823195 18298 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:22.839286 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:23.738628 18300 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:22.829790 18371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.005s
W20260709 12:18:23.745353 18371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.007s	sys 0.010s
W20260709 12:18:22.997403 18372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1882) took 2259 ms (client timeout 1764 ms). Trace:
W20260709 12:18:22.987735 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 202953us
I20260709 12:18:23.753078 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2114) took 1498 ms. Trace:
I20260709 12:18:22.976387 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:23.856841 18440 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.981156 18444 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:18:23.862059 18444 pending_rounds.cc:85] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 9
I20260709 12:18:23.862319 18444 pending_rounds.cc:107] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:18:22.978015 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:23.864133 18447 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:23.864317 22625 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.864858 22625 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.867028 22625 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:23.867388 22665 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.867975 22673 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.867939 22665 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.868626 22673 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.870349 22665 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:23.870786 22673 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:23.185935 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 448571us
I20260709 12:18:23.124866 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:22.979337 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:23.877780 22673 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.878309 22673 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.880594 22673 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:23.078683 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.884357 22723 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.792577 19039 raft_consensus.cc:614] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:23.435381 18296 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:23.889266 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2364) took 1700 ms. Trace:
I20260709 12:18:23.889621 18296 rpcz_store.cc:276] 0709 12:18:22.188933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.189538 (+   605us) service_pool.cc:224] Handling call
0709 12:18:22.189857 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.887481 (+1697624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.889021 (+  1540us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:23.889204 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2835880832}
I20260709 12:18:23.465344 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:23.488348 18366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1894) took 2345 ms (client timeout 2106 ms). Trace:
I20260709 12:18:23.489941 18367 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 12. Ops: []
W20260709 12:18:23.407800 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45824.9us
I20260709 12:18:23.558257 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:23.891904 18369 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:23.617192 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:23.335300 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:23.597903 18443 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:23.893149 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:23.893505 18368 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.893899 19038 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:23.894095 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:23.892993 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 908) took 1436 ms. Trace:
I20260709 12:18:23.894850 19035 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:23.637324 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 895) took 2446 ms. Trace:
I20260709 12:18:23.895640 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 897) took 2246 ms. Trace:
W20260709 12:18:23.890960 18366 rpcz_store.cc:269] 0709 12:18:21.143061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.155923 (+ 12862us) service_pool.cc:224] Handling call
0709 12:18:23.488037 (+2332114us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54000c85d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:23.488244 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1270347648}
I20260709 12:18:23.154567 22599 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:23.898351 19042 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:23.898309 22599 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.898741 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:23.621914 22251 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:22.773579 22661 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:23.904227 19037 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:23.902982 18368 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:23.905727 22470 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:18:23.906085 18368 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 912) took 1448 ms (client timeout 1568 ms). Trace:
W20260709 12:18:23.906677 18368 rpcz_store.cc:269] 0709 12:18:22.457595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.314980 (+857385us) service_pool.cc:224] Handling call
0709 12:18:23.906026 (+591046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.543573 22644 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:23.295185 22684 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:23.911176 22684 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.537613 22727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.001s	sys 0.002s
W20260709 12:18:23.914001 22727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.147s	user 0.004s	sys 0.003s
I20260709 12:18:23.554807 22690 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.947389 22690 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:23.576152 22725 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.426s	user 0.006s	sys 0.000s
I20260709 12:18:23.553647 22694 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:23.949755 22690 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:23.954743 22694 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.957085 22694 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:22.820065 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:23.959649 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 1.63859e+06us
I20260709 12:18:23.960534 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:23.702630 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:23.931488 22759 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:18:23.961593 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.543s (SENT)
I20260709 12:18:23.961880 22703 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:23.962733 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:23.963511 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 369882us
I20260709 12:18:23.963857 22782 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.739117 18300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 766) took 2404 ms (client timeout 1526 ms). Trace:
I20260709 12:18:23.754814 22655 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:23.966902 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:23.754925 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:23.971869 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 217010us
I20260709 12:18:23.973081 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:23.764032 18371 rpcz_store.cc:276] 0709 12:18:22.254041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.254174 (+   133us) service_pool.cc:224] Handling call
0709 12:18:22.254449 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.291560 (+ 37111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.291588 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.292030 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.745797 (+1453767us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:23.745848 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.752409 (+  6561us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.752549 (+   140us) raft_consensus.cc:1685] finished
0709 12:18:23.752601 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.752933 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36783,"raft.queue_time_us":37823,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"thread_start_us":1453374,"threads_started":1,"wal-append.queue_time_us":80718,"wal-append.run_cpu_time_us":1681,"wal-append.run_wall_time_us":1060650}
I20260709 12:18:23.795419 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 907) took 1339 ms. Trace:
I20260709 12:18:23.802958 22490 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.803216 22571 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.803975 18396 debug-util.cc:398] Leaking SignalData structure 0x7b080016ca20 after lost signal to thread 17160
I20260709 12:18:23.975399 18370 rpcz_store.cc:276] 0709 12:18:22.456285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.619720 (+1163435us) service_pool.cc:224] Handling call
0709 12:18:23.620065 (+   345us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:23.620499 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.620533 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:23.621401 (+   868us) log.cc:844] Serialized 78 byte log entry
0709 12:18:23.663044 (+ 41643us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:23.663481 (+   437us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.663919 (+   438us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.664104 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.664196 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.795146 (+130950us) raft_consensus.cc:1685] finished
0709 12:18:23.795166 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.795341 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":310,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":184005,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":251,"thread_start_us":40862,"threads_started":1,"wal-append.queue_time_us":162603}
W20260709 12:18:23.978794 18396 debug-util.cc:398] Leaking SignalData structure 0x7b080016c620 after lost signal to thread 18232
W20260709 12:18:23.979184 18396 debug-util.cc:398] Leaking SignalData structure 0x7b080016bce0 after lost signal to thread 18246
I20260709 12:18:23.979750 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:23.980340 18396 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6a60 after lost signal to thread 18324
W20260709 12:18:23.980628 18396 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6fa0 after lost signal to thread 18325
I20260709 12:18:23.981076 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:23.854095 22776 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:23.861560 22190 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.964663 18445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2461) took 2128 ms (client timeout 1543 ms). Trace:
I20260709 12:18:23.863346 22769 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.873106 22648 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.991523 22773 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.991382 22648 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.990676 22776 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.992127 22773 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.993868 22648 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:23.875275 22685 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.994146 22776 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:23.994343 22773 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:23.994741 22776 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:23.885404 22612 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:23.995656 22612 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.636045 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 904) took 1353 ms. Trace:
I20260709 12:18:23.996523 18375 rpcz_store.cc:276] 0709 12:18:22.282615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.317482 (+ 34867us) service_pool.cc:224] Handling call
0709 12:18:23.273575 (+956093us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:23.274096 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.274138 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:23.274933 (+   795us) log.cc:844] Serialized 52 byte log entry
0709 12:18:23.629885 (+354952us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:23.635426 (+  5541us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.635638 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.635714 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.635753 (+    39us) raft_consensus.cc:1685] finished
0709 12:18:23.635766 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.635973 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":978,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":145194,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":186,"thread_start_us":319089,"threads_started":2,"wal-append.queue_time_us":342978}
I20260709 12:18:23.997835 22612 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:23.895985 19043 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:23.998272 22660 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:23.998438 22612 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:23.990764 18445 rpcz_store.cc:269] 0709 12:18:20.836339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.836532 (+   193us) service_pool.cc:224] Handling call
0709 12:18:22.964353 (+2127821us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54002ab054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:22.964551 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399601920}
I20260709 12:18:23.897524 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 910) took 1440 ms. Trace:
I20260709 12:18:23.552994 22445 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.005612 22445 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.891139 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1964) took 1234 ms. Trace:
I20260709 12:18:23.887041 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 21) took 1108 ms. Trace:
I20260709 12:18:24.007711 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.007880 19039 rpcz_store.cc:276] 0709 12:18:22.778553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.784662 (+  6109us) service_pool.cc:224] Handling call
0709 12:18:23.886952 (+1102290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.900362 18444 raft_consensus.cc:2889] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.902376 22251 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.009670 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002) took 1032 ms. Trace:
I20260709 12:18:24.010146 18444 rpcz_store.cc:276] 0709 12:18:22.977044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.980492 (+  3448us) service_pool.cc:224] Handling call
0709 12:18:22.980802 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.009319 (+1028517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.009597 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.603189 22645 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.016820 22645 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.910517 22599 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:23.894260 22625 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.911877 22684 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.027884 22684 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:23.592113 22640 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.030393 22645 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.030460 22640 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.032231 22445 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.032692 22640 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:23.895783 18373 rpcz_store.cc:276] 0709 12:18:21.191288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.202082 (+ 10794us) service_pool.cc:224] Handling call
0709 12:18:21.202395 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.237545 (+ 35150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:21.237591 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.782686 (+1545095us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b2c0005b678. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:18:22.784099 (+  1413us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.821292 (+ 37193us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:22.821350 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.821449 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.636869 (+815420us) raft_consensus.cc:1685] finished
0709 12:18:23.636896 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.637131 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16071,"raft.queue_time_us":55509,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7039,"spinlock_wait_cycles":9077632,"thread_start_us":36746,"threads_started":1,"wal-append.queue_time_us":851214}
I20260709 12:18:23.913473 22663 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.896037 18367 rpcz_store.cc:276] 0709 12:18:21.649190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.650718 (+  1528us) service_pool.cc:224] Handling call
0709 12:18:23.489708 (+1838990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.891391 (+401683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.895473 (+  4082us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54000c85d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:23.895586 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4189846784}
I20260709 12:18:23.930570 18447 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:23.931815 18369 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:24.034093 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1004) took 1056 ms. Trace:
W20260709 12:18:24.034207 18369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 909) took 1577 ms (client timeout 2068 ms). Trace:
I20260709 12:18:23.932780 18440 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:24.034473 18447 rpcz_store.cc:276] 0709 12:18:22.977635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.977745 (+   110us) service_pool.cc:224] Handling call
0709 12:18:24.034029 (+1056284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.939172 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:24.035161 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1000) took 1064 ms. Trace:
I20260709 12:18:23.934291 22780 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:24.035586 18442 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.959731 22657 raft_consensus.cc:2764] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:24.033564 18443 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:23.964627 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.034641 18369 rpcz_store.cc:269] 0709 12:18:22.456829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.557910 (+1101081us) service_pool.cc:224] Handling call
0709 12:18:24.034144 (+476234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:24.038511 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.183s (SENT) [suppressed 38 similar messages]
W20260709 12:18:23.703336 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.751107 18372 rpcz_store.cc:269] 0709 12:18:20.738480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.738625 (+   145us) service_pool.cc:224] Handling call
0709 12:18:22.997152 (+2258527us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400168ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:22.997286 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502669056}
W20260709 12:18:23.965394 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 262853us
I20260709 12:18:24.035983 22780 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:22.971573 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:23.968631 22470 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.060055 22470 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:24.060253 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001) took 1089 ms. Trace:
I20260709 12:18:24.060467 22470 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.060628 18441 rpcz_store.cc:276] 0709 12:18:22.971122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.971283 (+   161us) service_pool.cc:224] Handling call
0709 12:18:24.060189 (+1088906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.970044 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:23.973073 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:23.981629 18396 debug-util.cc:398] Leaking SignalData structure 0x7b080038bde0 after lost signal to thread 18399
I20260709 12:18:24.062779 18448 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:18:24.063714 18396 debug-util.cc:398] Leaking SignalData structure 0x7b08003eaf80 after lost signal to thread 18473
I20260709 12:18:24.064275 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:24.065277 18396 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7100 after lost signal to thread 19068
W20260709 12:18:24.065913 18396 debug-util.cc:398] Leaking SignalData structure 0x7b0800326f00 after lost signal to thread 22378
W20260709 12:18:24.066468 18396 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5a40 after lost signal to thread 22638
W20260709 12:18:24.066895 18396 debug-util.cc:398] Leaking SignalData structure 0x7b0800232700 after lost signal to thread 22651
I20260709 12:18:24.067540 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:23.832055 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 770) took 2496 ms. Trace:
W20260709 12:18:23.835831 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.984067 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51022.3us
I20260709 12:18:23.985139 22786 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:23.986771 22787 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.979710 22490 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.983400 22655 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:18:23.988044 22190 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.979517 22694 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.090770 22694 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.091172 22694 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.092381 22788 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:24.083312 22660 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:18:24.093026 22788 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.991250 22769 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:23.983209 22571 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.079459 22786 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.095748 22786 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:23.877166 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:23.884889 22723 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.994624 22685 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:24.096887 22571 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:23.982283 22782 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.097795 22723 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.098517 22685 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.098763 22640 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:24.098897 22782 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.099087 22788 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:23.895020 18374 rpcz_store.cc:276] 0709 12:18:22.456569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.616920 (+1160351us) service_pool.cc:224] Handling call
0709 12:18:23.892932 (+276012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.003443 18366 rpcz_store.cc:276] 0709 12:18:22.457104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.896774 (+1439670us) service_pool.cc:224] Handling call
0709 12:18:23.897042 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.897208 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.897235 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.897276 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:23.897316 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.897367 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.897466 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.102321 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.007401 18291 rpcz_store.cc:276] 0709 12:18:22.656633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.465004 (+808371us) service_pool.cc:224] Handling call
0709 12:18:23.891078 (+426074us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:24.024484 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49982.9us
I20260709 12:18:24.028654 18449 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:24.029089 22625 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.033471 22663 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:23.945250 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 911) took 1487 ms. Trace:
I20260709 12:18:24.035527 18440 rpcz_store.cc:276] 0709 12:18:22.970832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.976093 (+  5261us) service_pool.cc:224] Handling call
0709 12:18:24.035099 (+1059006us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:23.965564 18300 rpcz_store.cc:269] 0709 12:18:21.334740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.337696 (+  2956us) service_pool.cc:224] Handling call
0709 12:18:22.839085 (+1501389us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b5800003698. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:23.739019 (+899934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17671936}
I20260709 12:18:24.042033 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:24.119922 18368 rpcz_store.cc:276] 0709 12:18:22.457351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.907389 (+1450038us) service_pool.cc:224] Handling call
0709 12:18:23.907720 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.907928 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:23.908231 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.944858 (+ 36627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.944916 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:23.944977 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.945050 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.945184 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1552,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":105,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":33536,"thread_start_us":32160,"threads_started":1,"wal-append.queue_time_us":32543}
W20260709 12:18:24.121156 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 79221.3us
I20260709 12:18:24.120425 18443 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:24.050522 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.051613 18373 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:24.057581 18442 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.041879 22657 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.082610 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.126571 18371 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:24.127635 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:24.128173 19038 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:24.128466 19038 raft_consensus.cc:614] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:24.084764 18372 pending_rounds.cc:85] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 4
I20260709 12:18:24.129714 18372 pending_rounds.cc:107] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:24.130061 18372 raft_consensus.cc:2889] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:24.087286 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.087875 22655 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:24.132965 22655 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:24.133172 18298 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 9: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:24.133780 22655 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.087929 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.081037 22773 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.135469 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 258370us
I20260709 12:18:24.135560 18300 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:18:24.136247 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 148342us
I20260709 12:18:24.046816 22645 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.062198 22785 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:18:24.089938 18296 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:24.147765 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.090279 22787 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.092490 22490 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.156226 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:23.998737 22784 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:23.634697 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2125) took 1085 ms. Trace:
I20260709 12:18:24.162026 18446 rpcz_store.cc:276] 0709 12:18:22.548993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.549119 (+   126us) service_pool.cc:224] Handling call
0709 12:18:22.549468 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.590433 (+ 40965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.590480 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.591027 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.711526 (+120499us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:22.711564 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.634263 (+922699us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54001a6654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:23.634316 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.634366 (+    50us) raft_consensus.cc:1685] finished
0709 12:18:23.634382 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.634627 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46320512,"thread_start_us":120092,"threads_started":1,"wal-append.queue_time_us":128680,"wal-append.run_cpu_time_us":2762,"wal-append.run_wall_time_us":1022065}
I20260709 12:18:24.082085 22251 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.124056 22782 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:24.169070 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.068626 18299 rpcz_store.cc:276] 0709 12:18:21.335838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.339571 (+  3733us) service_pool.cc:224] Handling call
0709 12:18:23.120707 (+1781136us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b5800003698. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:18:23.120958 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.121210 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:23.121580 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.830303 (+708723us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.830667 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.831481 (+   814us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.831546 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:23.831623 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.831713 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.831965 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320547,"prepare.queue_time_us":4753,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":122,"raft.queue_time_us":329787,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":222,"spinlock_wait_cycles":13498624,"thread_start_us":387349,"threads_started":3,"wal-append.queue_time_us":492853}
I20260709 12:18:24.170851 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:24.125288 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 86800.1us
I20260709 12:18:24.136178 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:24.089118 18294 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:24.137575 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.135764 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.090204 22190 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.148422 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.148484 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.188802 18443 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:24.149190 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.149534 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.149884 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.150159 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.139297 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.140290 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.151796 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.151589 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.138324 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.096817 22769 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.145038 18371 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:24.154645 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.151589 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:23.983579 22690 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:24.089144 22780 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.124221 22703 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.105230 19039 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:24.235858 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 99751us
I20260709 12:18:24.167804 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.236826 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.167635 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.134960 22788 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.173513 19035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:24.147716 22571 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:24.176062 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 206065us
I20260709 12:18:24.242362 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:24.135416 22684 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:24.175374 19041 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:24.175863 18299 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:24.099298 22640 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.237469 19044 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:24.167949 22251 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.247665 18299 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:24.238494 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:24.239034 18198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 294) took 19561 ms (client timeout 15000 ms). Trace:
W20260709 12:18:24.189426 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 116749us
I20260709 12:18:24.236538 18293 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:24.237660 18292 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:24.231724 18449 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.241123 18297 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:24.237668 18291 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:24.240629 18442 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:24.250177 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:24.249460 19043 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:24.244442 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.151569 22645 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.255164 18368 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:24.255563 18368 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:24.248486 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76390.2us
W20260709 12:18:24.226174 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90945.5us
I20260709 12:18:24.175546 22470 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.240813 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:24.253677 18293 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:18:24.153357 22773 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.240442 22665 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.264496 18293 pending_rounds.cc:85] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 9
I20260709 12:18:24.264715 19040 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:24.263693 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.264999 18293 pending_rounds.cc:107] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:18:24.266716 18293 raft_consensus.cc:2889] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:24.267343 18448 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.267428 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.268255 18440 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:24.242393 22490 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.241909 22571 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.138813 22657 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.260723 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.224725 22723 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:18:24.250998 22251 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.252089 22685 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:24.283629 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.245066 22782 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:18:24.248220 22690 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:24.254743 22648 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:24.282300 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.279191 22571 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.250252 22684 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.295116 19036 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:18:24.294263 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:24.296003 18300 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:24.299830 22792 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:24.283739 18368 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.301069 22645 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.301905 18297 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:24.275578 22490 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.260653 22780 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.311322 18375 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:24.286721 22776 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:18:24.311674 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:24.312914 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 70675us
I20260709 12:18:24.314742 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.284076 18446 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:24.244580 22788 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:24.305282 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 108108us
I20260709 12:18:24.434402 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:24.315712 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 46434.9us
I20260709 12:18:24.273010 22190 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.285256 22774 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.286945 22665 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:24.271371 22789 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:24.242749 22703 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.399410 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.274597 22644 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:24.428223 18368 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.258922 22791 raft_consensus.cc:2764] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:24.319932 22283 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:24.317582 22685 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:24.435766 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:24.436322 18441 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
W20260709 12:18:24.436960 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.434075 18373 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:24.442822 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 128135us
W20260709 12:18:24.249830 18198 rpcz_store.cc:269] 0709 12:18:04.607846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.608038 (+   192us) service_pool.cc:224] Handling call
0709 12:18:24.169538 (+19561500us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:20.170767 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:20.178206 (+  7439us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:20.178284 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:18:20.178333 (+    49us) tablet.cc:662] Decoding operations
0709 12:18:20.180112 (+  1779us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:20.180222 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:20.180262 (+    40us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:18:20.181733 (+  1471us) tablet.cc:701] Row locks acquired
0709 12:18:20.181783 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:18:20.182027 (+   244us) write_op.cc:270] Start()
0709 12:18:20.182172 (+   145us) write_op.cc:276] Timestamp: P: 1783599500181915 usec, L: 0
0709 12:18:20.182194 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:18:20.183048 (+   854us) log.cc:844] Serialized 5582 byte log entry
0709 12:18:20.609969 (+426921us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:20.871661 (+261692us) op_driver.cc:464] REPLICATION: finished
0709 12:18:21.189405 (+317744us) write_op.cc:301] APPLY: starting
0709 12:18:21.189511 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:21.874776 (+685265us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:21.874840 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:22.344183 (+469343us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:22.379686 (+ 35503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:18:22.383139 (+  3453us) write_op.cc:312] APPLY: finished
0709 12:18:22.411681 (+ 28542us) log.cc:844] Serialized 129 byte log entry
0709 12:18:22.500562 (+ 88881us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:22.500729 (+   167us) write_op.cc:454] Released schema lock
0709 12:18:22.500798 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:22.501273 (+   475us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:22.501308 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":304825,"apply.run_cpu_time_us":30025,"apply.run_wall_time_us":1311829,"cfile_cache_hit":54,"cfile_cache_hit_bytes":56376,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":1,"lbm_read_time_us":841,"lbm_reads_lt_1ms":8,"num_ops":14,"prepare.queue_time_us":388991,"prepare.run_cpu_time_us":28377,"prepare.run_wall_time_us":199211,"raft.queue_time_us":196602,"raft.run_cpu_time_us":11457,"raft.run_wall_time_us":326167,"replication_time_us":651586,"spinlock_wait_cycles":4937472,"thread_start_us":795377,"threads_started":5,"wal-append.queue_time_us":361765,"wal-append.run_cpu_time_us":25230,"wal-append.run_wall_time_us":2306763}]]}
I20260709 12:18:24.427664 22490 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.453761 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.440861 22470 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.438156 22774 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.458873 22774 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.464560 22685 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.465777 22784 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.466198 22784 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.433131 18196 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 10 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:24.429948 22802 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:24.437269 22445 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:24.476554 18196 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 6 to 10, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 10 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:24.484699 18375 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:24.499773 18196 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 8 to 9, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:24.447856 22283 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.507210 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.450779 22771 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.468814 22789 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:24.515241 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 220849us
I20260709 12:18:24.437531 22780 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.487990 22648 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:24.441452 22190 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.437985 22795 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:18:24.487952 22801 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:24.502525 22802 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:24.454159 22644 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.535269 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.512274 22283 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:24.526137 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 367753us
I20260709 12:18:24.549258 18196 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:24.515101 22771 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.504117 22445 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.525359 22648 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.534128 22782 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:24.553366 22470 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:24.644302 22727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.004s	sys 0.014s
I20260709 12:18:24.559816 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.566238 22787 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:24.560714 22800 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:18:24.560587 22684 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.571740 22802 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.564647 22490 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.572871 22655 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:24.576380 22774 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.659006 22796 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:18:24.680281 22782 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:24.553038 22640 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:24.571009 22798 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:18:24.506369 22690 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:24.581274 22625 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.581434 22251 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.559893 22780 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.585421 22786 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.678472 22794 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.586663 22657 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.536558 22665 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:24.698596 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed260 after lost signal to thread 17160
I20260709 12:18:24.589824 22703 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.591564 22789 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.702188 22806 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:18:24.581287 22788 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.591809 22784 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.539913 22776 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:18:24.595537 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.595666 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.596179 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.558482 22808 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 12:18:24.604454 22694 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.000s
I20260709 12:18:24.612773 22190 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.512039 22804 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:18:24.627660 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 184272us
I20260709 12:18:24.643220 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.645403 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.646776 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:24.662690 18196 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:24.732044 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.560822 22773 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.673728 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:24.677414 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 243080us
I20260709 12:18:24.677338 18367 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.680327 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.639755 22599 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.681308 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 213398us
I20260709 12:18:24.639462 22772 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:24.651667 22769 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.690027 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.993s (SENT)
I20260709 12:18:24.692041 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.662147 22648 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:24.651834 22771 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:24.679998 22727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.004s	sys 0.015s
I20260709 12:18:24.788760 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:24.694365 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:24.694922 17162 debug-util.cc:398] Leaking SignalData structure 0x7b080031f720 after lost signal to thread 18338
I20260709 12:18:24.680154 22470 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.691094 22795 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:24.696959 22799 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.763165 22788 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.717520 18296 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:24.717926 19035 raft_consensus.cc:606] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:24.808419 19035 raft_consensus.cc:614] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:24.719336 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800054880 after lost signal to thread 18232
W20260709 12:18:24.727581 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 211111us
I20260709 12:18:24.729758 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:24.809044 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800319140 after lost signal to thread 18246
I20260709 12:18:24.809475 18446 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:24.732057 19044 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:24.810155 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa720 after lost signal to thread 18324
W20260709 12:18:24.736893 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 479408us
W20260709 12:18:24.753334 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 108049us
W20260709 12:18:24.810464 19065 debug-util.cc:398] Leaking SignalData structure 0x7b080017c520 after lost signal to thread 18325
W20260709 12:18:24.811563 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800339d00 after lost signal to thread 18399
I20260709 12:18:24.812131 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.789047 22640 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:18:24.813256 22776 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:24.813373 22625 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:24.813678 22776 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:24.813755 22773 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:24.814640 22773 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:24.814787 22625 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:24.814570 22776 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.815207 22625 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:24.813510 22673 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:18:24.816069 22283 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.816015 22625 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:24.812928 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800354940 after lost signal to thread 18473
I20260709 12:18:24.815814 22776 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.817459 22625 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.818717 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:24.819357 19044 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:24.760756 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.821389 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800400240 after lost signal to thread 19039
I20260709 12:18:24.765336 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:24.822132 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002ea040 after lost signal to thread 19068
I20260709 12:18:24.818437 22283 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:24.779517 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 89544.7us
I20260709 12:18:24.779541 18368 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:24.780457 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:24.834159 18444 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.778951 18196 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 11, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:24.756851 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:24.835791 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 79041.2us
I20260709 12:18:24.837164 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.702940 22490 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.782945 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.784966 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.686189 22571 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.784621 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:24.789345 18441 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:24.842316 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57749.6us
I20260709 12:18:24.791435 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.709224 22774 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.791503 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:24.730751 22804 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:24.717202 22251 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.848503 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.864480 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.864938 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.707617 22445 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.717680 22786 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.540268 22663 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.797143 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.776834 22190 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:24.753034 22793 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.799896 22470 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:24.780215 22660 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:18:24.927659 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.804454 22799 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.805150 22795 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.758622 22806 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:24.709893 22782 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.847358 22645 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:18:24.808171 22788 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.817799 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:24.822427 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6e20 after lost signal to thread 19069
I20260709 12:18:24.822207 18443 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
W20260709 12:18:24.964481 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd4e0 after lost signal to thread 22510
W20260709 12:18:24.964820 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd300 after lost signal to thread 22625
I20260709 12:18:24.757102 18373 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:24.824281 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:24.965235 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800391a40 after lost signal to thread 22662
I20260709 12:18:24.964967 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:24.965672 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800385b80 after lost signal to thread 22700
I20260709 12:18:24.824815 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:24.966056 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003e4f00 after lost signal to thread 22716
I20260709 12:18:24.965044 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:24.966358 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08004122e0 after lost signal to thread 22717
W20260709 12:18:24.966611 19065 debug-util.cc:398] Leaking SignalData structure 0x7b080030a800 after lost signal to thread 22722
I20260709 12:18:24.966647 18440 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:24.966961 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800305ca0 after lost signal to thread 22729
W20260709 12:18:24.967242 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002864c0 after lost signal to thread 22730
W20260709 12:18:24.967496 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1540 after lost signal to thread 22735
W20260709 12:18:24.967744 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003b86a0 after lost signal to thread 22740
W20260709 12:18:24.967990 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08001d6c60 after lost signal to thread 22744
W20260709 12:18:24.968221 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800209e60 after lost signal to thread 22745
I20260709 12:18:24.839299 18369 raft_consensus.cc:2376] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10. Re-sending same reply.
I20260709 12:18:24.833407 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.839385 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.824576 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.847443 18196 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:24.963725 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 145983us
I20260709 12:18:24.849254 18367 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:24.849706 19036 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:24.970237 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:24.970528 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:24.851948 19039 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:24.865516 18372 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.752547 22694 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.866406 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:24.919271 22801 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:24.876588 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 85216.5us
I20260709 12:18:24.779080 22785 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:18:24.865537 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.979628 18445 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:18:24.979950 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 114469us
I20260709 12:18:24.980613 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:24.900458 18441 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.903861 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:24.779660 22599 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:24.841778 22673 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.988165 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 149377us
I20260709 12:18:24.915354 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.919739 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.989282 18375 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:24.989454 18371 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:24.863813 22657 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.787931 22769 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:24.867780 22640 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:24.847163 22703 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:24.924610 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 133169us
I20260709 12:18:24.921068 22684 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:24.999811 22684 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.000289 22684 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:24.994732 22445 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.000852 22445 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:24.994485 22645 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.001654 22645 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:25.002029 22645 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.949263 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:24.902081 22283 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:24.978379 22694 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.987735 22727 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:24.968477 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003aef00 after lost signal to thread 22751
I20260709 12:18:24.849675 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:25.018834 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08000b36e0 after lost signal to thread 22753
I20260709 12:18:24.870860 19044 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:24.872793 22773 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:18:24.872673 22780 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 12:18:24.974730 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 141408us
W20260709 12:18:24.977488 22818 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:18:24.977806 22644 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:24.978336 22831 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:24.978639 22795 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:24.968792 22794 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.986831 18442 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:24.990072 19043 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:24.982260 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:24.987752 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:25.000877 22799 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:25.042659 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54977us
I20260709 12:18:24.960041 22470 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.997690 22788 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.001279 22445 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.947276 22804 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.006479 18372 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.955890 18444 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:24.980976 22801 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.784446 22772 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.014465 18374 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:18:25.031163 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.028203 18367 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:24.996361 22640 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:25.040863 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.008185 22652 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.997565 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:25.011490 22283 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:25.068589 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 94930.3us
W20260709 12:18:25.045334 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 125295us
I20260709 12:18:25.200485 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:25.070777 22795 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.996713 22673 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.038946 22663 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.067468 18196 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:25.028990 22703 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:25.057376 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 75199.7us
I20260709 12:18:25.056300 22794 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.037199 22831 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:25.190366 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 122544us
W20260709 12:18:25.196743 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 199241us
I20260709 12:18:25.017386 22251 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.073812 22801 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.073643 22814 raft_consensus.cc:2764] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:25.194759 22640 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.217331 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:25.235162 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:25.207420 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 218141us
W20260709 12:18:25.233425 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.237576 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.235163 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:25.201973 22283 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.238705 19044 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:25.202129 22789 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:25.069459 22470 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.189338 22825 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:25.237811 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:25.241879 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.258288 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.258960 18371 raft_consensus.cc:2376] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11. Re-sending same reply.
I20260709 12:18:25.242542 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.244477 22251 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.245824 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.188292 22791 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:18:25.191438 22571 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.204984 22795 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.198261 22804 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.249784 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:25.255446 22812 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 12:18:25.186125 22771 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:25.260681 18374 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:25.219323 22794 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:25.294232 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60559.9us
I20260709 12:18:25.204313 22657 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:25.203809 22652 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.309255 19043 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:25.297706 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:25.256314 22829 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.252955 22789 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.256021 22820 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:25.308990 22648 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 12:18:25.320184 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:25.320456 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:25.311857 22655 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.004s
I20260709 12:18:25.314568 22785 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.326431 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:25.310616 22799 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:25.327759 18196 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 7, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:25.366465 18196 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 6 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:25.368577 18196 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 5 to 7, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:25.330302 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:25.317890 22821 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 12:18:25.309262 22769 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.310429 22795 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:25.334954 22834 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:18:25.336185 22673 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.336603 22772 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.337632 22470 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.338001 22251 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:25.339980 22827 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:18:25.340976 22828 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:18:25.304255 22813 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.437134 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 110764us
I20260709 12:18:25.344612 18441 raft_consensus.cc:2368] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:25.440050 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:25.326045 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:25.311520 22825 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:25.358124 18447 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:18:25.362862 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.362186 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 331333us
I20260709 12:18:25.360466 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:25.435036 18371 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:25.442828 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 82457.9us
I20260709 12:18:25.444751 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:25.416334 18196 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 4 to 11, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:25.445431 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:25.445815 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:25.333266 22791 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.415123 18440 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:25.443078 18371 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:25.445010 19039 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:25.332652 22789 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:25.454558 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:25.446831 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:25.448141 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:25.424768 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:25.427456 22826 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:18:25.433508 22840 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:18:25.374614 22799 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:25.432915 22833 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:18:25.455855 19043 raft_consensus.cc:606] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:25.422143 22599 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:25.450454 18371 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.456764 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:25.457008 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:25.459146 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:25.460841 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.451884 22773 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:25.449908 22813 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.463827 22673 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.464470 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:25.464716 22858 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:25.451731 22644 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.449136 22791 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:25.359555 22648 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.372380 22788 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:18:25.421267 22703 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:18:25.448442 22251 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.472373 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:25.479260 22799 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:25.478678 22829 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.482406 18444 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:25.356446 22820 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.327150 22831 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:25.484999 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 163581us
I20260709 12:18:25.486166 22652 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.487552 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:25.487581 18371 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.490410 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.497972 19043 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:25.474339 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:25.530725 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:25.531535 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:25.532604 18444 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:25.533825 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:25.499212 18300 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.534845 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:25.483093 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:25.488417 22679 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:25.497280 22625 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.003s
I20260709 12:18:25.504933 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:25.500883 18292 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:25.538067 18291 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:25.575767 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 70894.7us
I20260709 12:18:25.540979 18300 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.575662 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:25.499193 22251 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:25.575139 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.499544 22673 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:25.540010 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52543.3us
I20260709 12:18:25.502408 22645 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.579813 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:25.544638 22679 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.598178 18295 raft_consensus.cc:2410] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:18:25.594686 19044 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:25.580828 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:25.539296 22644 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:25.598749 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.603931 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.606477 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.608446 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:25.609067 18292 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:25.579375 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.610113 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.610832 18295 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:25.603310 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.610946 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.578179 22856 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.612946 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.597929 22694 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:25.658910 22694 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.659436 22694 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.595649 22791 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.660305 22791 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.621714 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.578323 22571 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.626204 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.627028 22673 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.602299 22831 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:25.628373 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.669569 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.669790 18372 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:25.628949 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.620034 18440 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:25.628949 22784 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.634294 22648 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.635036 19037 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:25.602406 22772 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.637272 22283 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:25.638008 22684 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.637s	user 0.007s	sys 0.000s
I20260709 12:18:25.642441 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:25.646257 22445 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.510s	user 0.000s	sys 0.007s
W20260709 12:18:25.647585 22852 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:18:25.539381 22599 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:18:25.647820 22813 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:25.648890 22776 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.831s	user 0.003s	sys 0.007s
I20260709 12:18:25.649016 22820 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.595548 22251 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.578441 22819 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:25.650365 22645 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.607368 22679 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.609841 22771 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:25.651891 22644 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.538714 22847 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:18:25.538908 22825 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.656314 22640 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:25.657433 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.657569 22856 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.657447 22857 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:25.665249 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:25.593860 18371 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:25.695168 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.698793 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.741425 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:25.742128 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 152119us
I20260709 12:18:25.704005 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.711618 18298 raft_consensus.cc:2368] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12.
I20260709 12:18:25.695492 22858 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:25.741837 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:25.700726 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58355.4us
W20260709 12:18:25.743667 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.744266 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.744984 22794 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.745111 18339 leader_election.cc:400] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:25.742448 18371 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:25.740876 18295 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:25.756670 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:25.759058 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:25.759357 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:25.760542 18444 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:25.703052 22859 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.701005 22684 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.701807 22772 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.764176 18443 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:25.765558 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.770344 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:25.772154 18295 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:25.702950 22445 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.767386 22657 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.777969 22858 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.780998 22657 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.781448 22657 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:25.784021 18338 connection.cc:256] RPC call timeout handler was delayed by -1.8442779289998725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:18:25.784426 18338 connection.cc:256] RPC call timeout handler was delayed by -1.7398779289999311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 12:18:25.752331 22645 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:25.784840 18338 connection.cc:256] RPC call timeout handler was delayed by -1.721877928999902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
I20260709 12:18:25.757917 22813 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.786788 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:25.787918 18338 connection.cc:256] RPC call timeout handler was delayed by -1.6534779289997914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 12:18:25.767707 22847 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:25.792003 18372 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:18:25.789220 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3996779289998358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 12:18:25.792562 18441 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
W20260709 12:18:25.793677 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3978779289998329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
W20260709 12:18:25.794370 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2988779289999002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 12:18:25.794790 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2943779289998929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
I20260709 12:18:25.797428 18292 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
W20260709 12:18:25.795410 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2844779289998769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 12:18:25.799198 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2691779289998522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 12:18:25.791304 22859 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:25.800997 22859 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.801404 22859 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:25.807610 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.733510 22819 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.819581 18372 pending_rounds.cc:85] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:25.820055 18372 pending_rounds.cc:107] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:25.820343 18372 raft_consensus.cc:2889] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:25.795888 22847 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:25.837356 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:25.823097 22850 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.250s	user 0.004s	sys 0.002s
I20260709 12:18:25.846133 22799 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.822566 22870 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:18:25.790406 22645 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.866045 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.754868 22857 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:25.844722 22690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.005s	sys 0.004s
I20260709 12:18:25.792299 22813 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:25.844519 22847 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.884567 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.889711 18294 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:18:25.876881 22690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.006s	sys 0.004s
I20260709 12:18:25.896230 18294 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:25.900622 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.903098 18370 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:25.906440 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.904551 18446 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:25.912232 22645 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.907919 22684 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:25.913364 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:25.908533 22772 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.899405 22283 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.921034 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.907960 22445 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.933966 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.937602 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.903414 22655 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:18:25.943018 22859 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:25.944762 22868 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:18:25.946774 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:25.953576 18371 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:25.950891 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.920073 22640 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:25.956874 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 913) took 1360 ms. Trace:
I20260709 12:18:25.957269 18367 rpcz_store.cc:276] 0709 12:18:24.596839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.994970 (+398131us) service_pool.cc:224] Handling call
0709 12:18:25.008910 (+ 13940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.193648 (+184738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.956631 (+762983us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b54000f9454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:25.956815 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17523584}
I20260709 12:18:25.958427 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.958487 22679 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:25.961043 22640 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:25.949060 22858 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:25.960421 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.960042 22645 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.961912 18297 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.954485 18338 leader_election.cc:400] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9.
I20260709 12:18:25.959004 18367 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:25.972662 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:25.933132 22794 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:25.997349 18338 leader_election.cc:403] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:26.016309 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:26.017119 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62804.8us
I20260709 12:18:26.017232 18368 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:26.017947 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.019330 18444 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:26.018795 22793 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 9.
I20260709 12:18:26.022879 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.024145 22857 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:26.022068 22789 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.003s
I20260709 12:18:26.027310 22862 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.027500 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.031931 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.032656 18367 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:26.032931 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.030876 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.033675 18370 raft_consensus.cc:2376] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10. Re-sending same reply.
I20260709 12:18:26.034088 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.036351 22856 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:26.037318 22820 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:25.975543 22858 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:26.047049 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.037351 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.052774 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.054314 22793 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.054391 22799 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.055377 22819 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.056413 22861 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:18:26.059180 22861 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:26.059847 22865 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:18:26.054689 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.061332 18370 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:26.062793 18296 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:26.039139 22870 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.039431 22794 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.017349 22640 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.077105 22251 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.079572 22873 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:26.079710 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:26.080703 22873 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.081089 22873 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:26.080705 18295 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:26.081050 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:26.082469 18294 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:26.067313 22859 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.088271 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.101008 18300 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:26.102205 18295 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:26.103256 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:26.103966 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.035220 22789 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.103725 22663 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.108665 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:26.109337 18295 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:26.109908 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:26.110170 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:26.111366 22866 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:18:26.113279 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:26.114642 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:26.114738 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:26.120838 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:26.122113 18367 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:26.122013 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 225852us
I20260709 12:18:26.121443 18291 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:26.070431 22793 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:18:26.129256 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:26.133143 19039 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:26.107759 22813 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.138586 22844 raft_consensus.cc:2764] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Leader election decision vote started in defunct term 11: won
I20260709 12:18:26.141534 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.142735 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.142634 22694 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.146317 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1025) took 1995 ms. Trace:
I20260709 12:18:26.146687 19042 rpcz_store.cc:276] 0709 12:18:24.150618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.150733 (+   115us) service_pool.cc:224] Handling call
0709 12:18:24.151050 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.659031 (+507981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.659060 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.660950 (+  1890us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.135777 (+474827us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:25.310436 (+174659us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.313016 (+  2580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.313133 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.144198 (+831065us) spinlock_profiling.cc:243] Waited 9.33 ms on lock 0x7b1800209380. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:26.146006 (+  1808us) raft_consensus.cc:1685] finished
0709 12:18:26.146134 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.146258 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2292,"prepare.queue_time_us":169838,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":180896,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":56184,"spinlock_wait_cycles":21468544,"thread_start_us":1016673,"threads_started":3,"wal-append.queue_time_us":762160}
I20260709 12:18:26.149003 22873 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.149884 22694 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.150493 22873 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.135429 22829 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.157866 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:26.159442 18373 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:18:26.159929 18373 pending_rounds.cc:85] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:26.160218 18373 pending_rounds.cc:107] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:18:26.160498 18373 raft_consensus.cc:2889] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:26.134625 22801 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.881s	user 0.010s	sys 0.002s
I20260709 12:18:26.128756 19043 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:26.169939 18373 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:26.162786 18444 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:26.165406 22792 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:26.165760 22802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.035s	sys 0.073s
W20260709 12:18:26.166661 22885 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:18:26.167012 22875 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:18:26.167902 18370 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:26.171200 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:26.181777 18370 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:26.173272 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.183483 18368 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:26.184074 18444 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.185531 18373 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:26.163285 22780 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.108187 22858 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:26.178141 22802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.477s	user 0.035s	sys 0.073s
I20260709 12:18:26.174749 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:26.191445 22807 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.202600 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.203209 18373 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:26.176997 22792 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.202121 18441 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:26.210700 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:26.213776 18441 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:26.208855 18444 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:26.176357 22801 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:26.217445 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:26.200785 22640 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.202397 22807 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:26.211349 22792 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.198638 22780 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:26.228379 18448 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:26.217738 22858 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:26.232666 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.231020 22789 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:26.249994 22645 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.250814 22780 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.251986 22780 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.253448 22657 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.250223 18291 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:26.255052 18371 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:26.235744 22893 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:26.237854 22251 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:26.266412 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:26.267164 18291 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:26.267976 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:26.269001 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:26.269635 22831 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.000s
I20260709 12:18:26.270395 18291 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 12:18:26.271981 22801 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:26.274904 18373 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:26.258633 22807 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:26.275877 22807 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:26.285241 22795 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.851s	user 0.010s	sys 0.000s
I20260709 12:18:26.285704 22795 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.286777 18440 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:26.279099 22893 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:26.289449 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.261159 22817 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.123s
I20260709 12:18:26.291045 22893 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:26.303817 18415 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 2.997s (SENT)
W20260709 12:18:26.304718 18415 leader_election.cc:336] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 3.058s (SENT)
I20260709 12:18:26.306629 22795 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:26.308866 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:26.306918 19009 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.568s (SENT)
I20260709 12:18:26.312933 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:26.312860 22792 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.314072 22645 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.309576 22873 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:26.316474 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.317205 18443 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.306469 22847 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.322317 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.325083 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.427919 22785 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:26.471897 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 166443us
I20260709 12:18:26.472034 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:26.472229 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 189569us
I20260709 12:18:26.473101 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.474635 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.475750 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.476905 18375 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:26.460115 22857 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.355834 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.461414 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.407912 22190 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.464121 18291 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:26.425201 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:26.326231 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.477420 18443 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:26.481304 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56280.9us
I20260709 12:18:26.478060 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.457106 22657 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:26.482774 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 156601us
I20260709 12:18:26.487641 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.491508 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.491834 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.493010 19039 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:26.493716 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.495827 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.500344 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.505913 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.502054 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.516409 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.517416 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.519412 22791 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.519589 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:26.520478 22894 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:18:26.520768 22640 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.503175 22893 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.521288 22640 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:26.521924 22640 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.522758 22801 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:26.503348 22645 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:26.523191 22801 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:26.523661 22801 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:26.507797 22780 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:26.508294 22657 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:26.488741 22190 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.529580 22190 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.502885 22814 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.497246 22795 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.486908 22652 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.497928 22820 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.487020 22873 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:26.520213 22791 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:26.534188 22652 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.545222 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.543682 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:26.540496 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:26.549592 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.545002 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:26.553656 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:26.486730 22857 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.558300 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:26.563846 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.554901 22655 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:18:26.549196 22870 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:26.548118 22825 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:26.539944 22652 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.561609 22831 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:26.594110 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 48984.8us
I20260709 12:18:26.584589 22789 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:26.588408 22788 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.606524 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.594177 22829 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.627653 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:26.561174 22795 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:18:26.583057 22780 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:18:26.627560 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.561471 22814 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:26.598394 22862 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:26.632169 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:26.584769 22785 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:18:26.597609 22792 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:26.597672 22870 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:26.626299 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:26.632946 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 151024us
W20260709 12:18:26.635509 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.623817 22663 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:18:26.598793 22831 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.654318 18446 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.654919 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:26.628273 22829 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:26.620069 22796 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:18:26.640307 22789 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.634924 22808 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:18:26.670318 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:26.639936 22785 consensus_queue.cc:1243] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:26.669256 22831 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.694191 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:26.654400 22870 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:26.698824 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 145250us
I20260709 12:18:26.669034 22470 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:26.705528 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:26.710371 18446 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:26.712057 22796 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:26.712613 22796 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.713043 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 808) took 1133 ms. Trace:
I20260709 12:18:26.713461 18299 rpcz_store.cc:276] 0709 12:18:25.579767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.579909 (+   142us) service_pool.cc:224] Handling call
0709 12:18:25.580203 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.591665 (+ 11462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.591710 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.295518 (+703808us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b2c000cd878. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:18:26.295961 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.534720 (+238759us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:26.534756 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.534827 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.712751 (+177924us) raft_consensus.cc:1685] finished
0709 12:18:26.712771 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.712982 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15037,"raft.queue_time_us":11840,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7558,"spinlock_wait_cycles":12509440,"thread_start_us":238449,"threads_started":1,"wal-append.queue_time_us":335376}
I20260709 12:18:26.714704 22796 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.715822 22831 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.717947 22831 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:26.720682 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:26.722959 18446 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:26.686813 22829 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.689628 22847 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:26.740671 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 85851.4us
I20260709 12:18:26.741515 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:26.738328 22896 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:18:26.840799 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 999) took 1309 ms. Trace:
I20260709 12:18:26.741778 22776 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.741071 22770 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:18:26.849809 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 107996us
I20260709 12:18:26.752398 22857 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.841245 19044 rpcz_store.cc:276] 0709 12:18:25.440061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.447915 (+  7854us) service_pool.cc:224] Handling call
0709 12:18:26.749489 (+1301574us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54004a56d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.749597 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282826112}
I20260709 12:18:26.846477 22825 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:26.845708 22907 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:26.848573 22190 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.845510 22776 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:26.891995 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2164) took 2117 ms (client timeout 2735 ms). Trace:
W20260709 12:18:26.894222 18442 rpcz_store.cc:269] 0709 12:18:24.774388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.795836 (+ 21448us) service_pool.cc:224] Handling call
0709 12:18:26.891748 (+2095912us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b54003557d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.891921 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14425344}
I20260709 12:18:26.875330 22912 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:26.894409 22907 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:26.941792 22907 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:26.943449 22909 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:18:26.880223 22825 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.947094 22825 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:26.905067 22912 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:26.947873 22912 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:26.949167 22807 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:26.949553 22807 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:26.950363 22807 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:26.951705 18447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1016) took 2876 ms (client timeout 2211 ms). Trace:
W20260709 12:18:26.952127 18447 rpcz_store.cc:269] 0709 12:18:24.075521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.075635 (+   114us) service_pool.cc:224] Handling call
0709 12:18:26.951533 (+2875898us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003b1fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.951634 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812940672}
I20260709 12:18:26.963862 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2488) took 1642 ms. Trace:
I20260709 12:18:26.964289 18448 rpcz_store.cc:276] 0709 12:18:25.321212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.344967 (+ 23755us) service_pool.cc:224] Handling call
0709 12:18:25.345205 (+   238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.229932 (+884727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.963598 (+733666us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:26.963802 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030047616}
W20260709 12:18:26.968761 22889 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.932s	user 0.005s	sys 0.010s
I20260709 12:18:26.970407 18369 pending_rounds.cc:85] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 7
I20260709 12:18:26.970742 18369 pending_rounds.cc:107] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:18:26.971002 18369 raft_consensus.cc:2889] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:26.919910 22892 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.705s	user 0.003s	sys 0.000s
I20260709 12:18:26.972158 22795 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.972760 22795 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.975384 22599 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:26.976922 22819 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:26.921428 19044 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:26.922036 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1990) took 1300 ms. Trace:
W20260709 12:18:26.923137 22854 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.350s	user 0.015s	sys 0.021s
I20260709 12:18:26.925043 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 1980) took 1328 ms. Trace:
W20260709 12:18:26.929064 22881 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.973s	user 0.002s	sys 0.000s
W20260709 12:18:26.930160 22878 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.900s	user 0.000s	sys 0.010s
I20260709 12:18:26.933645 22801 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:26.934060 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 1982) took 1336 ms. Trace:
I20260709 12:18:26.935063 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 1968) took 1032 ms. Trace:
I20260709 12:18:27.009186 18298 rpcz_store.cc:276] 0709 12:18:25.597250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.603724 (+  6474us) service_pool.cc:224] Handling call
0709 12:18:26.933862 (+1330138us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54000a9bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.934006 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237170432}
I20260709 12:18:27.009560 18371 rpcz_store.cc:276] 0709 12:18:25.902404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.960204 (+ 57800us) service_pool.cc:224] Handling call
0709 12:18:26.934879 (+974675us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54000f8a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.935011 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654015360}
I20260709 12:18:26.936808 22776 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.940732 22679 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:26.940529 22870 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:26.977701 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:26.921411 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:26.975019 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:26.978538 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.025209 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.025969 18371 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:26.983839 18373 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 12:18:26.985066 18292 rpcz_store.cc:276] 0709 12:18:25.621540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.625915 (+  4375us) service_pool.cc:224] Handling call
0709 12:18:26.921823 (+1295908us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:26.921922 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294792576}
I20260709 12:18:26.994275 18297 rpcz_store.cc:276] 0709 12:18:25.596663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.692758 (+ 96095us) service_pool.cc:224] Handling call
0709 12:18:25.693126 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.016212 (+323086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.924819 (+908607us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540004c4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:26.924988 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617730944}
I20260709 12:18:27.028146 19035 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:27.028946 22652 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:27.031653 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56721.9us
I20260709 12:18:27.034795 18449 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:27.035871 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:27.037360 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:27.038410 18446 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:27.037500 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:27.039027 18447 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.045604 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.051700 18448 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:26.985212 22807 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.052112 18449 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:27.053506 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:27.074788 18445 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:27.063551 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.038381 22640 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 12:18:27.077948 18442 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:27.058175 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 352781us
I20260709 12:18:27.076524 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.080276 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.081171 22694 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:27.081785 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:27.084549 22814 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.085054 22893 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:27.054359 22788 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:27.090240 22880 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.040s	user 0.000s	sys 0.009s
I20260709 12:18:26.978504 22829 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.098899 18447 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:27.038767 22847 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.103576 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2387) took 1501 ms. Trace:
I20260709 12:18:27.048950 22780 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:27.134581 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 464319us
I20260709 12:18:27.105057 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2167) took 1463 ms. Trace:
I20260709 12:18:27.135324 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.039587 22820 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
W20260709 12:18:27.100296 22878 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:18:27.108978 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.111996 22817 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.112818 22893 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:27.113095 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1037) took 2294 ms. Trace:
I20260709 12:18:27.113579 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:27.115149 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 130072us
I20260709 12:18:27.115660 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.116526 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.147405 19040 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:27.148023 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.148602 19044 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.117343 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.118702 19042 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:27.055033 22655 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.119568 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.120450 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.048799 22599 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.007117 22819 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.105544 22791 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:18:27.126868 22822 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.086s	user 0.000s	sys 0.020s
I20260709 12:18:27.135422 18372 rpcz_store.cc:276] 0709 12:18:25.641631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.793553 (+151922us) service_pool.cc:224] Handling call
0709 12:18:25.819350 (+ 25797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.820991 (+  1641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.821035 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.821939 (+   904us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.892123 (+ 70184us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:25.892154 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.892223 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.928536 (+1036313us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:27.092857 (+164321us) raft_consensus.cc:1685] finished
0709 12:18:27.092886 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.104966 (+ 12080us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14513920,"thread_start_us":69740,"threads_started":1,"wal-append.queue_time_us":133307}
I20260709 12:18:27.039429 22795 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.141566 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.146675 22802 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:27.143206 19041 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:27.100445 22694 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.101739 22490 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.101552 22652 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.121495 22679 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.142192 22847 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.077291 22796 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:18:27.128069 18300 rpcz_store.cc:276] 0709 12:18:25.602629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.610741 (+  8112us) service_pool.cc:224] Handling call
0709 12:18:27.103331 (+1492590us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:27.103504 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125214080}
I20260709 12:18:27.165019 19037 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
W20260709 12:18:27.181770 22822 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.130s	user 0.000s	sys 0.020s
I20260709 12:18:27.201092 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:27.148624 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 924) took 2411 ms. Trace:
I20260709 12:18:27.163877 22470 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.142531 22788 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.102173 22640 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.031412 22858 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.142187 22780 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.176335 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:27.146610 22893 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:27.211194 22893 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.212064 22829 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.212551 22893 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.125069 22814 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.216342 22663 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:27.216868 22663 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:27.217566 22814 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.183168 19044 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:27.147032 19038 rpcz_store.cc:276] 0709 12:18:24.818988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.819129 (+   141us) service_pool.cc:224] Handling call
0709 12:18:24.819445 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:24.819793 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.819819 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:24.820385 (+   566us) log.cc:844] Serialized 52 byte log entry
0709 12:18:24.976517 (+156132us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:25.296792 (+320275us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.301546 (+1004754us) spinlock_profiling.cc:243] Waited 965 us on lock 0x7b1800273700. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287
0709 12:18:26.738188 (+436642us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.111871 (+373683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.112332 (+   461us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.112840 (+   508us) raft_consensus.cc:1685] finished
0709 12:18:27.112858 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.113019 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244838,"prepare.queue_time_us":476318,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":4175,"raft.queue_time_us":348161,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":63103,"spinlock_wait_cycles":2219904,"thread_start_us":1036849,"threads_started":5,"wal-append.queue_time_us":377640,"wal-append.run_cpu_time_us":3798,"wal-append.run_wall_time_us":1470456}
I20260709 12:18:27.219506 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.219990 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.221122 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.221165 22470 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.222235 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.222988 19038 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:27.128065 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:27.223289 22470 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:27.188445 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.150337 22810 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:27.223572 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 95576.8us
I20260709 12:18:27.223659 22652 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:27.224172 22652 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:27.224037 22810 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.224557 22652 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.225888 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.187212 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.226100 22810 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.201335 18372 pending_rounds.cc:85] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:18:27.226653 19042 raft_consensus.cc:2368] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:27.226841 18372 pending_rounds.cc:107] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:27.227223 18372 raft_consensus.cc:2889] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:27.202065 18366 rpcz_store.cc:276] 0709 12:18:24.736973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.027057 (+290084us) service_pool.cc:224] Handling call
0709 12:18:25.027369 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.249636 (+222267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.250056 (+   420us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.250525 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.455364 (+204839us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:25.455400 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.455462 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.922601 (+1467139us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:27.142798 (+220197us) raft_consensus.cc:1685] finished
0709 12:18:27.142818 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.148562 (+  5744us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17439,"raft.queue_time_us":165377,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":17614,"spinlock_wait_cycles":18635136,"thread_start_us":334477,"threads_started":2,"wal-append.queue_time_us":188507}
I20260709 12:18:27.227312 22788 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.229446 22788 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.230051 22785 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:27.230955 22785 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:27.207194 22780 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.231328 22723 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:27.171631 22599 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.234180 22599 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:27.235457 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2492) took 1391 ms. Trace:
I20260709 12:18:27.235988 18440 rpcz_store.cc:276] 0709 12:18:25.843537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.843760 (+   223us) service_pool.cc:224] Handling call
0709 12:18:25.844040 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.287172 (+443132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.235145 (+947973us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54003b10d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:27.235374 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017766144}
I20260709 12:18:27.231709 22723 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:27.237226 22847 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.238018 22723 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.206063 22912 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.238765 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.239189 22723 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:27.187354 22490 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.241091 22490 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.197162 22251 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:18:27.226961 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:27.207518 18447 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:27.181298 22694 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.209380 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.210023 18449 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:27.210106 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:27.220091 19044 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:27.256887 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.257406 19044 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:27.258221 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.151692 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.249053 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:27.249527 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.259089 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:27.251796 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.252996 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.260476 18366 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:27.260516 18375 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:27.261356 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.261982 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.260038 19044 raft_consensus.cc:606] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.186977 22870 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.264153 18366 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:27.265833 18366 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:27.266300 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:27.268076 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:27.268788 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.269248 18297 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:27.269872 18441 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:27.270299 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.174314 22655 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.271353 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.278671 19042 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:27.271785 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.273864 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.273684 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.275072 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:27.276068 18292 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:27.276477 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.279845 18291 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:27.257874 22898 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.727s	user 0.000s	sys 0.006s
I20260709 12:18:27.284739 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.280901 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.205613 22802 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:27.282651 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.151060 22817 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.286419 18375 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.287899 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:27.289234 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.226671 22810 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.260284 22470 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:27.292593 22470 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:27.292222 22802 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.293462 22470 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.260974 22769 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.258168 22847 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.305424 18300 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
W20260709 12:18:27.337221 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 260807us
I20260709 12:18:27.337499 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:27.341570 19037 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:27.328241 18371 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:27.342078 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.344252 18371 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:27.344506 18300 pending_rounds.cc:85] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 9
I20260709 12:18:27.335683 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 932) took 1861 ms. Trace:
I20260709 12:18:27.345146 18300 pending_rounds.cc:107] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:27.345808 18300 raft_consensus.cc:2889] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:27.281809 22921 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:18:27.207149 22858 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.347580 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.353834 18300 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.353835 19042 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:27.356858 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:27.326277 22911 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.481s	user 0.005s	sys 0.000s
W20260709 12:18:27.372843 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 217047us
I20260709 12:18:27.306424 22644 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.327546 22190 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.263022 22916 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:18:27.254711 22815 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:18:27.362236 18291 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:27.289161 22907 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:27.345737 18374 rpcz_store.cc:276] 0709 12:18:25.473830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.474428 (+   598us) service_pool.cc:224] Handling call
0709 12:18:25.474714 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.592868 (+118154us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:25.593113 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.811576 (+218463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.811640 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:25.811694 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.335456 (+1523762us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540030ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:27.335497 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.335613 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167938,"prepare.queue_time_us":1781,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4618,"raft.queue_time_us":25795,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7235,"spinlock_wait_cycles":271282048,"thread_start_us":196250,"threads_started":1,"wal-append.queue_time_us":287520,"wal-append.run_cpu_time_us":2104,"wal-append.run_wall_time_us":1176975}
I20260709 12:18:27.326031 22918 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.379963 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.390028 18300 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:27.390903 22599 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.391997 22599 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.392418 18412 leader_election.cc:400] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:27.392846 18412 leader_election.cc:403] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:27.393606 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.394333 22599 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.394433 22795 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:27.329704 22819 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:27.337751 22873 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:27.395143 22819 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:27.395588 22819 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.395901 22873 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.395917 22819 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:27.396348 22873 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.398487 22873 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.349700 18375 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:27.307214 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.391358 22788 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.399660 22788 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.401522 22788 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.406502 22819 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.394876 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:27.413112 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:27.416097 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2377) took 1928 ms. Trace:
I20260709 12:18:27.416474 18296 rpcz_store.cc:276] 0709 12:18:25.487782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.488011 (+   229us) service_pool.cc:224] Handling call
0709 12:18:25.488328 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.063153 (+574825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.415862 (+1352709us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54002378d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:27.416028 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320743680}
I20260709 12:18:27.425349 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:27.434950 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:27.470026 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59177.1us
I20260709 12:18:27.470618 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.472777 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:27.476269 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1949) took 1675 ms. Trace:
I20260709 12:18:27.374979 22787 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.352397 22847 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.342869 22769 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.383033 22804 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:27.476742 18367 rpcz_store.cc:276] 0709 12:18:25.800710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.127319 (+326609us) service_pool.cc:224] Handling call
0709 12:18:26.136589 (+  9270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.155901 (+ 19312us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:26.155945 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.156551 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.260809 (+104258us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:26.260836 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.260882 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.214542 (+953660us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:27.416213 (+201671us) raft_consensus.cc:1685] finished
0709 12:18:27.416233 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.476162 (+ 59929us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24677,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":18617216,"thread_start_us":74669,"threads_started":1,"wal-append.queue_time_us":280634}
I20260709 12:18:27.384505 22793 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
W20260709 12:18:27.469097 22921 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:18:27.429169 22788 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:27.480698 22795 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:27.431437 22814 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:27.488662 22911 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:18:27.557196 22774 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:27.475030 22770 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:18:27.480388 22796 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:18:27.562189 22679 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:27.556470 22907 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:27.481122 22780 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.479167 22808 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:18:27.490263 22599 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:18:27.488493 22190 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:18:27.488626 22490 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 12:18:27.493180 22787 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.583686 22873 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:18:27.579771 22788 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.614697 22795 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.581800 22929 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:27.621110 22770 consensus_queue.cc:1243] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:27.648660 18448 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:27.583690 22470 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 12:18:27.658152 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 279253us
I20260709 12:18:27.640064 22774 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:27.642117 22819 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:27.662744 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 284359us
W20260709 12:18:27.660452 22807 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.000s
I20260709 12:18:27.650755 22817 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:27.652004 22788 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:27.657737 22785 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:27.671574 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 198865us
I20260709 12:18:27.658609 22655 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.687578 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 1984) took 1029 ms. Trace:
I20260709 12:18:27.689297 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.691066 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.698206 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.698834 19035 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:27.674458 22858 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.687983 18368 rpcz_store.cc:276] 0709 12:18:26.658291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.658455 (+   164us) service_pool.cc:224] Handling call
0709 12:18:26.658876 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.659337 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:26.850271 (+190934us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.055825 (+205554us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.056283 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.056440 (+   157us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:27.056776 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.336557 (+279781us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b1800435e20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:27.336721 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.336788 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.687389 (+350601us) raft_consensus.cc:1685] finished
0709 12:18:27.687412 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.687525 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277344,"prepare.queue_time_us":244507,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":908,"raft.queue_time_us":303012,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":19865,"spinlock_wait_cycles":5391616,"thread_start_us":360498,"threads_started":2,"wal-append.queue_time_us":289076}
I20260709 12:18:27.709483 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.685902 22820 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:27.709873 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.710103 18292 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:27.698133 22923 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.312s	user 0.001s	sys 0.000s
I20260709 12:18:27.712821 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.726881 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:27.713806 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.721498 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:27.722955 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.688871 22807 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:27.725221 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 577206us
I20260709 12:18:27.725661 19040 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:18:27.687412 22934 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:27.728367 18375 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:27.730198 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:27.732394 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1005) took 1658 ms. Trace:
I20260709 12:18:27.732558 19044 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Advancing to term 11
I20260709 12:18:27.732806 19043 rpcz_store.cc:276] 0709 12:18:26.074285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.074486 (+   201us) service_pool.cc:224] Handling call
0709 12:18:26.100830 (+ 26344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.115064 (+ 14234us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:26.280995 (+165931us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.660805 (+379810us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.930978 (+270173us) spinlock_profiling.cc:243] Waited 950 us on lock 0x7b18000c6940. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:27.088465 (+157487us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18000c6940. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:27.088624 (+   159us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.089124 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.089265 (+   141us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:27.089300 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.089372 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.325667 (+236295us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:27.732189 (+406522us) raft_consensus.cc:1685] finished
0709 12:18:27.732214 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.732338 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":224441,"prepare.queue_time_us":15376,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":468299,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":45355,"spinlock_wait_cycles":45338368,"thread_start_us":323091,"threads_started":1,"wal-append.queue_time_us":420589}
I20260709 12:18:27.679816 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:27.732360 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 425180us
W20260709 12:18:27.739173 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 329247us
I20260709 12:18:27.740162 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:27.738296 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58584.5us
I20260709 12:18:27.727048 22819 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.723470 22820 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:27.727813 22655 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.754034 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:27.754879 19040 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:27.678159 22907 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.729542 22858 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:27.763345 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:27.753294 22769 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.753113 22847 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:27.731027 22934 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.770571 22942 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:18:27.728718 22774 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.771656 22780 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:27.785606 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51528.6us
W20260709 12:18:27.792650 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 413841us
W20260709 12:18:27.793754 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 323214us
I20260709 12:18:27.736217 22807 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.757557 22819 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.773584 22939 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:27.760094 22655 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.804327 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:27.795874 22950 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:27.810025 22931 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:27.810173 22950 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:27.811187 22950 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.809458 22953 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:18:27.809403 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:27.811781 22932 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:27.813820 18446 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:27.816277 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:27.775604 22820 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:27.803896 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 63636.1us
I20260709 12:18:27.820940 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.825086 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.828069 18291 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:27.825773 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.827389 18296 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:27.826198 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.831789 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.833819 22490 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:27.834046 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:27.839103 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80157.6us
I20260709 12:18:27.813422 22787 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.814297 22937 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.813695 22949 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.822758 22190 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:27.837364 22939 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:27.854831 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.856110 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.841503 22918 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:27.844574 22820 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.843001 22929 raft_consensus.cc:2764] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:27.852265 22937 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:27.866400 22940 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:27.866768 22918 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:27.854322 22939 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:27.867291 22954 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:18:27.861079 22942 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.871354 22957 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.873910 22655 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:27.853026 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:27.908339 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.859138 22817 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:27.853484 22787 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.852902 22190 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.910681 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.911350 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.911568 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.911778 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:27.912189 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:27.855644 22780 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 12:18:27.921378 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68598.3us
W20260709 12:18:27.922554 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66524.7us
W20260709 12:18:27.920259 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 88279.7us
I20260709 12:18:27.924705 18370 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 8: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:27.849054 22663 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.931663 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:27.924117 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 124350us
I20260709 12:18:27.933106 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.914202 22960 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.937295 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.945292 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.938216 22948 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:27.945844 22723 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.000s
I20260709 12:18:27.939606 22807 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.925390 22190 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.946841 19038 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:18:27.941421 22652 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.716s	user 0.007s	sys 0.003s
I20260709 12:18:27.947561 22652 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:27.949259 22694 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.000s
I20260709 12:18:27.949652 22652 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:27.941852 22945 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:27.950309 22945 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.923043 22817 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:27.938361 22663 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.951023 18293 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:27.935781 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.922650 22929 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.972178 18300 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:18:27.973052 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.973621 18299 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:27.950691 22945 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:27.961799 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:27.973300 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.988299 18292 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:27.973964 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.988662 22963 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:27.992436 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.993562 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:27.994781 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:27.980211 18366 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:27.996212 18448 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:27.996443 18366 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:27.922492 22954 consensus_queue.cc:1243] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:27.997223 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:27.998051 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:27.955787 22723 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:27.959872 22817 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:27.822928 22847 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:27.987957 22960 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.925020 22787 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:28.020556 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 204370us
I20260709 12:18:28.002295 18448 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:18:27.998704 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:27.919672 22942 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:28.017535 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67141.1us
I20260709 12:18:27.920583 22957 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.987643 22948 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:27.996407 22938 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:18:27.992234 22652 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.021878 18448 pending_rounds.cc:85] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:28.029780 18448 pending_rounds.cc:107] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:28.030085 18448 raft_consensus.cc:2889] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:27.949658 22694 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:27.997390 18366 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:27.997678 22780 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.021811 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:28.011065 22929 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:28.028030 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:28.013020 22954 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:28.031993 18373 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:28.013235 22934 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.016682 22819 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:28.010465 22660 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 12:18:28.042323 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 43697.9us
I20260709 12:18:28.033893 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.013115 22679 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:18:28.049131 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.050503 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.051641 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.051812 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.052529 18448 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:28.053478 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.054198 18448 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:28.054589 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.055482 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:28.056474 18299 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:28.057571 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.059111 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.062288 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.064539 18448 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:28.065655 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.066587 18448 raft_consensus.cc:2410] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:28.066339 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.068540 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.070377 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:28.075315 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.076182 18444 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:28.075892 19040 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:28.078352 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.074590 22947 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 12:18:27.953893 22939 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.080873 18449 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:28.081271 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.032732 22655 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.040186 22929 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:28.045425 22941 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:28.081753 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:28.040560 22954 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.986173 22918 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.086683 22819 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.087405 22694 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.034193 22958 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:28.068652 22723 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.092129 22190 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.110024 22929 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:28.095427 22964 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.095793 22970 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:28.095906 22962 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.166s
I20260709 12:18:28.096601 22815 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.099566 22937 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.099835 22873 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:28.100420 22951 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 12:18:28.101256 22952 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:18:28.081750 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:28.077461 22943 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:18:28.074786 22644 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 12:18:28.114730 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 86788.7us
I20260709 12:18:28.164561 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.166782 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:28.167563 18371 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.118037 22974 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.119084 22978 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:28.043627 22780 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:28.111901 22893 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.001s
I20260709 12:18:28.086196 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.117280 22954 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.149549 22723 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.155186 18292 raft_consensus.cc:2368] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:28.126358 22820 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:28.153249 22190 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.143692 22808 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:28.160887 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51335.5us
I20260709 12:18:28.162583 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:28.162978 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 766) took 1014 ms. Trace:
I20260709 12:18:28.176709 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:28.156339 22958 consensus_queue.cc:1243] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:28.183786 22978 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:28.283989 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:28.195462 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 109312us
I20260709 12:18:28.200074 22858 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:28.285418 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:28.210027 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:28.214169 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 773) took 1016 ms. Trace:
I20260709 12:18:28.218250 22912 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.286006 19038 rpcz_store.cc:276] 0709 12:18:27.198062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.252566 (+ 54504us) service_pool.cc:224] Handling call
0709 12:18:28.213989 (+961423us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:28.214109 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534356864}
W20260709 12:18:28.224432 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.241571 22801 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:28.252941 22251 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:28.287390 22251 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:28.288097 22251 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.288790 22251 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.254684 22612 raft_consensus.cc:993] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:28.255781 22802 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:28.116755 22941 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.291496 22802 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:28.292353 22802 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.293180 22941 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.293344 22802 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.263463 18442 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
W20260709 12:18:28.264380 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 99917.7us
I20260709 12:18:28.293769 18442 pending_rounds.cc:85] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:18:28.294041 18442 pending_rounds.cc:107] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:18:28.294265 18442 raft_consensus.cc:2889] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.146880 22950 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:18:28.295430 22950 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:28.295771 22950 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:28.296521 22950 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.156546 22980 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.297669 22950 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:28.298035 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.160552 22873 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:28.299000 22873 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.301190 18367 pending_rounds.cc:85] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:18:28.301532 18367 pending_rounds.cc:107] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:28.301775 18367 raft_consensus.cc:2889] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.302019 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:28.183939 22965 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:18:28.275911 19044 rpcz_store.cc:276] 0709 12:18:27.148169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.286719 (+138550us) service_pool.cc:224] Handling call
0709 12:18:27.732376 (+445657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.768272 (+ 35896us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:27.768319 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:27.768379 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.768429 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.768522 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.162766 (+394244us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54007e8bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:28.162904 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36740,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":6529,"spinlock_wait_cycles":1022028160}
I20260709 12:18:28.277436 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.188604 18371 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:28.262898 22976 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:28.303578 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.303788 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.270262 22820 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.269819 22962 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:28.309746 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.126612 22948 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.312714 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:28.337524 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60168.7us
I20260709 12:18:28.271974 22785 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.163141 22815 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.307301 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:28.313561 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.345114 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.351471 22976 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.367779 22976 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.367913 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1053) took 1110 ms. Trace:
I20260709 12:18:28.368212 22976 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.301009 22858 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:28.369122 22685 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:28.369647 22685 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:28.369814 22858 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.301182 22978 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:28.369059 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61890.3us
I20260709 12:18:28.368309 19035 rpcz_store.cc:276] 0709 12:18:27.257267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.705647 (+448380us) service_pool.cc:224] Handling call
0709 12:18:27.705927 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:27.712357 (+  6430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.712407 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:27.728195 (+ 15788us) log.cc:844] Serialized 52 byte log entry
0709 12:18:27.804336 (+ 76141us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.814363 (+ 10027us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.814565 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.814634 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.225875 (+411241us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b18004be620. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:28.362500 (+136625us) raft_consensus.cc:1685] finished
0709 12:18:28.367694 (+  5194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.367852 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6044,"prepare.queue_time_us":8612,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":12659,"raft.queue_time_us":41491,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7203,"spinlock_wait_cycles":25238272,"thread_start_us":47104,"threads_started":1,"wal-append.queue_time_us":102405}
I20260709 12:18:28.371063 22858 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:28.376441 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.300860 22190 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.345964 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 941) took 1205 ms. Trace:
I20260709 12:18:28.348390 22802 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:28.359279 22971 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:28.370565 22685 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.361572 22776 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.363196 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2005) took 1086 ms. Trace:
I20260709 12:18:28.380781 18297 rpcz_store.cc:276] 0709 12:18:27.277134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.277243 (+   109us) service_pool.cc:224] Handling call
0709 12:18:27.277741 (+   498us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:27.283689 (+  5948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.283716 (+    27us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:27.341354 (+ 57638us) log.cc:844] Serialized 558 byte log entry
0709 12:18:27.677693 (+336339us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.677737 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.677805 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.362862 (+685057us) raft_consensus.cc:1685] finished
0709 12:18:28.362892 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.363121 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18029,"raft.queue_time_us":8883,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":293697,"threads_started":1,"wal-append.queue_time_us":449941}
I20260709 12:18:28.330142 22820 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.384526 22825 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:28.384953 22825 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:28.385905 22825 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:28.387588 22954 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.389801 22820 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.314345 22801 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.379792 18369 rpcz_store.cc:276] 0709 12:18:27.140619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.213711 (+ 73092us) service_pool.cc:224] Handling call
0709 12:18:27.214038 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.622413 (+408375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.622455 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.622902 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.766543 (+143641us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.766594 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.766684 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.344164 (+577480us) raft_consensus.cc:1685] finished
0709 12:18:28.344190 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.345890 (+  1700us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132860,"raft.queue_time_us":181241,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":418425,"threads_started":2,"wal-append.queue_time_us":305382}
I20260709 12:18:28.344138 22956 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.313309 22983 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:28.195744 22974 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.338281 22942 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.349030 22973 raft_consensus.cc:1081] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:28.346114 22787 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.280808 22808 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.435175 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 940) took 1294 ms. Trace:
I20260709 12:18:28.435768 18372 rpcz_store.cc:276] 0709 12:18:27.140402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.182915 (+ 42513us) service_pool.cc:224] Handling call
0709 12:18:27.201038 (+ 18123us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:18:27.668400 (+467362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.668445 (+    45us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:18:27.669361 (+   916us) log.cc:844] Serialized 104 byte log entry
0709 12:18:27.823991 (+154630us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.824044 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.824138 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.434843 (+610705us) raft_consensus.cc:1685] finished
0709 12:18:28.434887 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.435100 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119517,"raft.queue_time_us":167837,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"thread_start_us":453041,"threads_started":2,"wal-append.queue_time_us":247258}
I20260709 12:18:28.418890 22956 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:28.448123 22801 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:28.406528 22785 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.441448 22819 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:28.464473 19042 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 8.
I20260709 12:18:28.457450 22956 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.467795 22820 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.478591 19040 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:28.483055 22954 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.484241 22954 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.484869 22954 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.391012 22976 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.490495 22976 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.454833 22957 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.475661 22858 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:28.511837 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 202156us
I20260709 12:18:28.475826 22820 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:28.508893 18372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:18:28.468492 22974 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.512965 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:28.529161 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:28.514384 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.462248 22819 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:28.529500 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 26) took 1193 ms. Trace:
I20260709 12:18:28.531623 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.553238 19037 rpcz_store.cc:276] 0709 12:18:27.335930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.337380 (+  1450us) service_pool.cc:224] Handling call
0709 12:18:28.529062 (+1191682us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b540066eca0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df9520 00007f3c68e76652 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:28.529360 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338736000}
I20260709 12:18:28.551043 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:28.561233 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:28.562659 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:28.580122 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.506520 22950 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:28.426003 22983 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.521539 22780 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.509505 22807 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:28.635393 22807 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.635979 22814 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:28.636468 22814 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:28.637748 22807 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.637588 22814 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.521325 22723 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.555523 22801 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:28.649199 22801 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:28.649785 22801 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:28.633021 22950 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:28.654222 22951 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:18:28.655313 22987 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:18:28.655927 22815 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.657370 22785 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.659576 22814 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:28.660698 22788 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.964s	user 0.007s	sys 0.003s
I20260709 12:18:28.660632 22978 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.661199 22788 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.663482 18375 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:28.664091 22808 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:28.665259 22788 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.665622 22983 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:28.634735 22984 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:18:28.587067 22979 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:28.647960 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.648231 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 97268.1us
I20260709 12:18:28.569424 22974 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:28.654963 22950 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.648342 22599 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:28.670400 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 73299.2us
I20260709 12:18:28.674604 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.675159 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.706535 19035 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:28.706707 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:28.707058 19037 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:28.675899 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:28.706090 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:28.707926 19036 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.707994 18371 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:28.676254 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:28.708623 18444 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.676518 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:28.677253 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.675159 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.710945 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.712270 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.698750 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:28.699620 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:28.714226 19040 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.707558 18370 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:28.717717 18447 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:28.718758 18446 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:28.719619 19038 pending_rounds.cc:85] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 7
I20260709 12:18:28.720003 19038 pending_rounds.cc:107] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:18:28.720297 19038 raft_consensus.cc:2889] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.734242 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.621989 22954 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:28.693559 22963 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.000s	sys 0.018s
W20260709 12:18:28.736752 22963 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.000s	sys 0.018s
I20260709 12:18:28.736757 18449 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:28.737097 22963 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:28.737942 22963 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.551285 22858 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.724131 18371 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:28.739091 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:28.740746 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69290.7us
I20260709 12:18:28.741292 22858 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.743727 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:28.746608 22785 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:28.702845 22918 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.019s
W20260709 12:18:28.755189 22918 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.000s	sys 0.020s
I20260709 12:18:28.755266 22993 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:28.683145 22807 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.638795 22960 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.759644 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:28.760372 18447 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:28.767206 18371 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:28.774415 18444 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:28.569633 22820 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:28.650760 22801 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.700296 22789 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:28.780838 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:28.786724 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:28.791006 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:28.793057 19044 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:28.717486 22978 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:28.795056 18372 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:28.795537 18372 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.799212 18447 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
W20260709 12:18:28.792048 22847 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.000s
W20260709 12:18:28.804271 22935 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:18:28.790302 22973 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:28.735317 22599 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:28.825723 22949 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.955s	user 0.005s	sys 0.006s
I20260709 12:18:28.775724 22960 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.775794 22807 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:28.735050 22819 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:28.850639 22873 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.551s	user 0.004s	sys 0.002s
I20260709 12:18:28.804124 22723 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:18:28.803442 22789 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.854482 22949 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.858613 22820 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:28.849272 22956 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:28.869978 22945 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.872s	user 0.012s	sys 0.000s
I20260709 12:18:28.865392 22873 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:28.901611 22945 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:28.914717 22817 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.889s	user 0.011s	sys 0.000s
W20260709 12:18:28.935389 22936 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.166s	user 0.004s	sys 0.001s
W20260709 12:18:28.799579 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.003739 22599 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.016510 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:29.024206 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.036460 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.037788 19044 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:29.038884 19043 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:29.039847 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.045753 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2395) took 1072 ms. Trace:
I20260709 12:18:29.045956 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.046241 22847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46944 (request call id 22) took 29086 ms. Trace:
I20260709 12:18:29.046751 19037 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.048307 22807 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.053524 22723 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.926847 22817 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:29.056180 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.059633 22858 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.054713 22723 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.060495 22858 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.062057 22956 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.063287 18366 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.064143 22962 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:29.064244 22723 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.065167 22789 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.064965 22962 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:29.067648 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.067873 22978 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.060664 22862 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.077353 22978 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.078253 22862 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.080152 22978 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.080638 18441 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:29.080978 22980 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:29.081560 22862 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.081856 22980 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.082780 22817 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:29.084379 22929 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.924s	user 0.016s	sys 0.000s
I20260709 12:18:29.043874 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1043) took 1781 ms. Trace:
I20260709 12:18:29.085122 22929 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.085340 18440 rpcz_store.cc:276] 0709 12:18:27.250593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.258266 (+  7673us) service_pool.cc:224] Handling call
0709 12:18:27.258652 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.258915 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.258950 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.264911 (+  5961us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.640489 (+375578us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:27.640530 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.640594 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.866638 (+1226044us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:29.031659 (+165021us) raft_consensus.cc:1685] finished
0709 12:18:29.031707 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.032125 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8217856,"thread_start_us":294797,"threads_started":1,"wal-append.queue_time_us":435052}
I20260709 12:18:29.087687 22844 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:29.088130 22844 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:29.089805 22844 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.046820 22847 rpcz_store.cc:276] 0709 12:17:59.959414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.959641 (+   227us) service_pool.cc:224] Handling call
0709 12:18:00.206951 (+247310us) log.cc:844] Serialized 506 byte log entry
0709 12:18:00.731748 (+524797us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b180001eb80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:29.045948 (+28314200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16836,"raft.queue_time_us":59638,"raft.run_cpu_time_us":2566,"raft.run_wall_time_us":248435,"spinlock_wait_cycles":18920192,"thread_start_us":521573,"threads_started":2,"wal-append.queue_time_us":296334,"wal-append.run_cpu_time_us":11126,"wal-append.run_wall_time_us":1412510}
I20260709 12:18:29.046285 18291 rpcz_store.cc:276] 0709 12:18:27.971180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.973139 (+  1959us) service_pool.cc:224] Handling call
0709 12:18:29.042126 (+1068987us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:29.043562 (+  1436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258839936}
I20260709 12:18:29.105880 22873 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.091213 22844 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.093384 22962 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.099996 22858 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.117599 22980 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.122041 19035 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:29.122632 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.124490 22640 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:29.125852 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86550.6us
I20260709 12:18:28.842145 22973 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.132853 22973 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:29.148588 18263 connection.cc:256] RPC call timeout handler was delayed by -1.0891280620000998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:18:29.161468 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.162621 18291 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:29.162824 19037 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:29.165196 18293 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:29.166319 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.167701 19036 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:29.185010 22663 raft_consensus.cc:2764] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:29.187726 22958 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.188618 22958 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.193161 22958 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.194288 18366 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:29.196635 22819 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:29.196797 22836 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 10 is higher.
I20260709 12:18:29.198482 22844 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.167933 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.168401 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.170583 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.171481 22960 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.175616 22685 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:29.179160 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.203704 18291 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:29.187657 22929 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.209506 22960 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.210417 22960 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.206103 18299 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:29.198951 22844 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.213096 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2179) took 1954 ms. Trace:
I20260709 12:18:29.214476 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.215184 22960 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.215876 18442 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:29.217151 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.213749 18366 rpcz_store.cc:276] 0709 12:18:27.253574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.267019 (+ 13445us) service_pool.cc:224] Handling call
0709 12:18:27.267356 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.980083 (+712727us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400200e20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7e1c0 00007f3c68e7f231 00007f3c68e7bd8f 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c
0709 12:18:29.061882 (+1081799us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400200e20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df1a1d 00007f3c68dd81a4 00007f3c68e5a447 00007f3c68e72fe7 00007f3c68e747de 00007f3c68e7eb49 00007f3c68e7f416 00007f3c68e7bd8f 00007f3c68e7b83e 00007f3c6de40419
0709 12:18:29.196272 (+134390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.207471 (+ 11199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898774144}
I20260709 12:18:29.219856 22825 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.219942 22679 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.220775 22679 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:18:29.222086 22975 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.921s	user 0.002s	sys 0.007s
I20260709 12:18:29.223186 22679 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.226085 22679 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:29.171063 22939 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.232872 22685 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.246006 18372 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.248068 22685 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:29.215216 22944 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.400s	user 0.000s	sys 0.012s
I20260709 12:18:29.250051 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 825) took 1905 ms. Trace:
I20260709 12:18:29.250423 18298 rpcz_store.cc:276] 0709 12:18:27.344102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.344278 (+   176us) service_pool.cc:224] Handling call
0709 12:18:27.344553 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.344853 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.344879 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.351683 (+  6804us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.660023 (+308340us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:27.660050 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.660128 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.212972 (+552844us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:29.214709 (+1001737us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b240017fd68. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dce09d 00007f3c68a0010e 00007f3c68a1788c 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221
0709 12:18:29.249649 (+ 34940us) raft_consensus.cc:1685] finished
0709 12:18:29.249675 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.249900 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":394,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6526,"spinlock_wait_cycles":125486848,"thread_start_us":249491,"threads_started":1,"wal-append.queue_time_us":396757}
W20260709 12:18:29.252734 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.256551 22946 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:29.257253 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.257726 22946 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:29.258183 22946 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.258240 23026 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.259069 23026 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.259598 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.261526 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.262099 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.239210 18299 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:29.262143 22970 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.229557 18300 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:29.243419 18294 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:29.264832 22970 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.265838 23026 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.243346 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.241921 22957 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.262357 18373 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:29.278707 22685 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:29.293720 22957 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.294274 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.295313 18298 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:29.296526 18440 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:29.290768 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.305231 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.306200 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:29.306777 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.307691 19040 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:29.308246 18442 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:29.308640 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.309386 19044 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:29.310431 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.311616 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.284916 22679 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.312165 22945 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.312877 22652 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.314091 22970 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.314877 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.307017 18375 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:29.315622 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.316720 19043 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:29.317123 18373 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:29.305341 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.319280 18297 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:29.302220 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.324966 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.328531 19036 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:29.329054 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:29.331418 18293 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.331816 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.332707 18295 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:29.324970 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:29.338711 22970 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:29.324870 22946 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.378473 22946 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:29.381693 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:29.382469 18370 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:29.384508 18372 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.386031 22970 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.386694 22970 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:29.387275 19038 raft_consensus.cc:606] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:29.387811 19038 raft_consensus.cc:614] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:29.387912 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.388652 18442 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:29.339985 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50436.4us
I20260709 12:18:29.390802 22802 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:29.391223 22802 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.392292 22802 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.393399 22802 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:29.340852 22949 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:29.397795 22949 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.399482 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.400004 22949 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.341284 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.348308 22655 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:29.354461 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:29.355754 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.362811 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.364842 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2401) took 1374 ms. Trace:
I20260709 12:18:29.368528 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.374522 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.375219 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.375648 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.377051 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:29.431280 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 779) took 2052 ms (client timeout 1838 ms). Trace:
I20260709 12:18:29.432214 22844 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.432807 22679 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.439482 18292 rpcz_store.cc:276] 0709 12:18:27.990346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.992183 (+  1837us) service_pool.cc:224] Handling call
0709 12:18:29.364420 (+1372237us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54001cd9d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:29.364659 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371936000}
I20260709 12:18:29.435932 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.437296 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.438277 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.444445 18299 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
W20260709 12:18:29.409049 23035 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.068s	user 0.003s	sys 0.000s
W20260709 12:18:29.438946 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46544.4us
I20260709 12:18:29.442253 18293 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:29.433429 22949 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.440730 19042 rpcz_store.cc:269] 0709 12:18:27.378751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.379728 (+   977us) service_pool.cc:224] Handling call
0709 12:18:29.410103 (+2030375us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400264654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:29.430945 (+ 20842us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477914368}
I20260709 12:18:29.445101 18375 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:29.446928 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.447057 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:29.453212 18296 raft_consensus.cc:2410] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:18:29.457485 18374 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
W20260709 12:18:29.460913 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 199046us
I20260709 12:18:29.441645 22655 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:29.460644 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.436084 22957 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:29.455439 22910 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.012s	sys 0.022s
I20260709 12:18:29.475173 18370 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:29.470362 18440 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:29.475860 18442 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:29.471834 19038 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:29.484979 18443 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:29.485324 22831 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:29.485688 22831 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.486163 18371 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:29.487062 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.488023 18447 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:29.488381 18444 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.489698 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.491320 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:29.486604 22831 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.500272 22831 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:29.507961 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.510062 18291 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:29.510367 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:29.469450 22910 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.013s	sys 0.022s
I20260709 12:18:29.511034 18298 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:29.515877 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.516109 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.516655 18293 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:29.516399 22949 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:29.517373 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 142423us
I20260709 12:18:29.518155 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.518652 22949 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.519306 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:29.519622 18299 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:29.519968 19044 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:29.520859 18295 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:29.521912 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:29.523303 19040 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:29.524582 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.525202 18442 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:29.526659 18448 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:29.526638 18444 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:29.527542 22939 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:29.525467 18445 raft_consensus.cc:2410] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:18:29.529109 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.529246 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.530026 18293 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:29.527449 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.470920 22844 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.538787 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.539544 18444 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:29.539589 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:29.540380 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:29.545030 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.545532 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.547917 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.550972 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:29.549598 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:29.553105 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:29.551970 19044 raft_consensus.cc:606] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:29.554719 19044 raft_consensus.cc:614] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:29.554394 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.550997 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.565233 19035 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:29.566111 18447 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:29.559960 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.573194 18374 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.576642 22663 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:18:29.578249 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51460.9us
W20260709 12:18:29.583683 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.584548 18374 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:29.537890 22939 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:29.592146 18371 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:29.561753 22844 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.595157 22663 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.601364 22470 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260709 12:18:29.602173 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.601492 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.603529 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.605114 18370 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:18:29.599841 18441 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:29.603384 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.595698 18368 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.618296 18442 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:29.602177 22829 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:29.619720 22829 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.620133 22807 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.620889 22807 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.622104 22829 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.623374 22770 raft_consensus.cc:2764] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:18:29.623287 22663 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.624413 22470 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:29.624775 22470 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.625687 22470 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.626880 22470 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:29.627161 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.627763 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:29.613564 18447 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:29.633096 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 98603.2us
I20260709 12:18:29.634387 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.635474 22962 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:29.636121 22962 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:29.637099 22962 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.639426 22807 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.660501 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:29.661690 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:29.662570 22970 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:29.667865 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68129.3us
I20260709 12:18:29.681483 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:29.706351 18367 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.708319 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:29.709585 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.717773 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 246005us
I20260709 12:18:29.691529 22929 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.691504 22679 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.725589 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91378.1us
W20260709 12:18:29.726020 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.656117 22844 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.738358 22914 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:29.739195 23009 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:29.740402 22912 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:29.740801 22844 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.730891 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.742174 18374 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:29.717437 22939 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:29.743432 22939 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.743579 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.743846 22939 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:29.744311 18448 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 7.
W20260709 12:18:29.745918 23037 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:29.729172 22945 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:29.747751 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.749491 22945 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:29.749536 22964 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:29.750142 22945 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.751148 22964 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:29.751684 22974 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.663713 22770 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:18:29.752322 22974 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.753178 22679 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.753652 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:29.754619 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:29.754782 22974 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.755429 22825 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:18:29.755627 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45337.5us
I20260709 12:18:29.756083 22825 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:29.757053 22825 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.757297 22907 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:29.758787 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.759406 22858 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:29.759723 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.760185 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.760362 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.761137 18444 raft_consensus.cc:2410] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:29.759874 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.761312 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.762084 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.757055 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:29.762893 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.763540 23010 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:29.763909 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.764096 23010 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.732015 22644 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:29.732391 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.732832 22694 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.763571 18296 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:29.765239 22694 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.765390 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.766124 22970 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.732992 22947 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:18:29.733589 22807 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:29.741155 22929 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.740626 22787 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:18:29.731803 18369 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:29.759867 22858 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:29.732107 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:29.785908 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.775454 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.786343 18299 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:29.786428 18370 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.752835 22770 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:29.787374 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.787921 18367 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:29.764564 23010 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.785741 22957 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.790465 22787 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:29.799330 22787 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:29.799499 22957 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.799965 18445 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:29.800338 22787 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.764892 22644 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:29.802027 22787 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.785567 22907 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:29.791205 22858 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.801604 22957 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.740769 22912 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:29.802325 18300 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:29.804131 22912 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.653215 22655 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.795310 22770 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:29.796079 22694 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.798000 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:29.787335 18448 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:29.811728 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.775548 22819 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:29.755429 22679 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:29.818131 22912 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:29.820336 22655 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.823752 23004 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:29.823979 22770 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.824846 23004 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:29.825528 22770 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.826414 22929 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:29.826750 23004 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:29.827214 23004 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:29.828122 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 61236.3us
I20260709 12:18:29.829049 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.784577 22974 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:29.816061 18445 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:29.816980 18338 leader_election.cc:400] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:29.833209 18338 leader_election.cc:403] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:29.835812 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.826300 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:29.851583 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 139388us
I20260709 12:18:29.831197 22685 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:29.832182 22663 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:29.813774 22644 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:29.859309 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.846952 22831 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:29.866693 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:29.870235 18445 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:29.861799 22819 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:29.875931 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.859038 22663 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:29.877599 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.879347 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.879668 22251 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:29.879945 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58103.9us
I20260709 12:18:29.876027 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.881713 22948 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:29.889741 18444 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:18:29.882771 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 70819.3us
I20260709 12:18:29.871694 22870 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:29.893692 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.897415 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:29.860898 22907 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:18:29.903441 22787 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:29.907721 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.883368 22829 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:29.888207 22929 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.889994 22663 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:29.911388 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:29.914897 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:29.828975 23004 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:29.891916 22948 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.921190 18293 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.882180 22945 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:29.904209 22819 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:29.923417 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 147195us
I20260709 12:18:29.909394 22914 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:18:29.908385 22644 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:29.922947 22948 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.936017 18293 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:29.911738 22829 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.928443 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:29.918105 22916 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:18:29.947209 18368 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:29.948084 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:29.901780 22831 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:29.945680 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 89448.9us
I20260709 12:18:29.942080 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:29.957471 22862 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:29.952199 22829 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:29.948069 22807 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.968122 22945 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:29.974656 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46335.1us
I20260709 12:18:29.974963 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:30.005295 18293 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:29.980892 22870 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:29.989883 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:30.007586 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:30.008572 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:30.018210 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 267953us
W20260709 12:18:30.022856 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66835.6us
I20260709 12:18:30.019163 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:30.019883 18338 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.507s (SENT) [suppressed 14 similar messages]
I20260709 12:18:30.011686 22929 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.034896 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.038164 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:30.037710 23022 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:18:30.039256 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:30.039958 18444 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:30.016992 22807 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:30.042135 22831 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:30.021519 22870 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:30.022233 22974 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.043193 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.043622 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:30.030616 22644 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:18:30.044234 18441 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:30.045573 23006 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:30.043377 22870 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.047096 22679 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:30.047662 22679 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:30.037827 22819 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.022609 22802 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.051019 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:30.030591 22694 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.987622 22939 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.049748 22807 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.033643 22946 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:30.071600 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:30.072218 18292 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:30.091699 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:30.098846 18292 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
W20260709 12:18:30.100126 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 84146.8us
W20260709 12:18:30.111147 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 79419us
I20260709 12:18:30.116089 18441 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:30.072841 22946 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.049527 22974 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.125720 22907 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.136852 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 112115us
I20260709 12:18:30.139901 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.117916 22831 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:30.155079 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.099946 22858 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.167906 18366 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
W20260709 12:18:30.169914 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.153045 22929 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:30.203203 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:30.133785 22946 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:30.151716 22939 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.215286 22945 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:30.215857 22945 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:30.216827 22945 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.217435 22858 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:30.219614 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58775.1us
I20260709 12:18:30.206023 22947 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.220400 22947 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.223470 22974 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.220463 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.200245 22802 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.229957 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:30.237488 18444 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:30.240139 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.137854 22679 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.248522 22831 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.151628 22907 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.208303 22685 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:30.222997 22947 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.241784 22844 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.260572 18375 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:30.262673 18375 pending_rounds.cc:85] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 5
I20260709 12:18:30.229759 22929 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:18:30.262980 18375 pending_rounds.cc:107] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:30.270658 18375 raft_consensus.cc:2889] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:30.265803 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146131us
I20260709 12:18:30.286372 18442 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:30.257596 22685 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.291512 23006 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:18:30.292490 22655 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:30.285135 22831 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.320303 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:30.320583 18292 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:18:30.271531 22663 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.296761 22655 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.280380 22844 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:30.356679 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 336809us
I20260709 12:18:30.358112 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.358254 18299 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:30.361989 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:30.374467 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 353562us
W20260709 12:18:30.382347 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.410975 18373 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:30.417443 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:30.295584 22948 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.423064 22970 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.423854 22970 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:30.425239 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.425130 22970 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.430914 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:30.433137 18374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.025s	sys 0.028s
W20260709 12:18:30.433687 18374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.025s	sys 0.028s
I20260709 12:18:30.304994 22945 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:18:30.449375 22844 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.455992 18441 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:30.308359 22831 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.353102 22655 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.442430 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.485116 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:30.491829 18441 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:30.512727 23004 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:30.435801 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 73876.7us
I20260709 12:18:30.520098 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:30.523449 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 485374us
I20260709 12:18:30.523777 18444 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:30.534309 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:30.538858 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:30.544876 22825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.008s	sys 0.013s
I20260709 12:18:30.532207 22802 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:30.541079 22870 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:30.535670 22858 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:30.532058 22694 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:30.536372 22929 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:30.521471 22829 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:30.586253 18447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.015s	sys 0.006s
W20260709 12:18:30.606925 18447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.015s	sys 0.006s
W20260709 12:18:30.552683 22962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.004s	sys 0.016s
W20260709 12:18:30.580464 22825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.008s	sys 0.013s
I20260709 12:18:30.539978 22946 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.597244 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:30.583489 22858 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:30.640337 18443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.024s
W20260709 12:18:30.653823 18443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.006s	sys 0.025s
W20260709 12:18:30.634642 22962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.004s	sys 0.017s
W20260709 12:18:30.637729 22819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.036s
I20260709 12:18:30.655053 22858 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.684293 23057 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:30.684626 23055 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:30.685002 23057 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:30.685559 23057 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:30.689507 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.690027 22858 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:30.668246 22819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.037s
I20260709 12:18:30.692494 22819 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.695292 22858 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.654088 22685 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.611759 22802 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.685294 23055 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.710632 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:30.712877 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.668200 22817 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.664050 22825 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:30.666115 22663 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.730569 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.740063 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:30.740190 18371 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:30.745566 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:30.753291 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.754132 19037 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:30.755528 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:30.758198 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.759253 19036 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:30.638303 22929 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.770058 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:30.780973 19040 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:30.771070 18370 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:30.778136 19035 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:30.783325 19035 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:30.778400 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.780010 19038 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 9: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:30.786707 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:30.799788 18440 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:30.770176 22947 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:30.739399 22957 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:30.817806 19044 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
W20260709 12:18:30.790176 23010 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.973s	user 0.009s	sys 0.004s
I20260709 12:18:30.793744 19035 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.825830 19035 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.792477 22831 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:30.741364 22825 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.711553 22694 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.743036 22817 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.816262 22947 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:30.853605 23063 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.861441 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:30.846935 19040 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:30.827486 22819 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.828913 23054 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:18:30.876542 22870 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:30.883785 23068 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:18:30.884795 22870 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.889827 22470 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.890439 22470 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.891324 22802 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.892302 22946 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.893631 22948 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:30.894604 23049 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:30.897121 23004 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:30.897725 23004 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.899852 23055 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.900245 22774 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:30.823033 23010 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:30.864727 22970 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.860976 22947 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:30.912715 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 327027us
W20260709 12:18:30.864741 18445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.029s	sys 0.045s
I20260709 12:18:30.824872 22957 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:30.916646 18445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.030s	sys 0.046s
W20260709 12:18:30.918370 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.878769 22829 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.863838 22655 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.928011 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:30.931337 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 114162us
I20260709 12:18:30.932853 19035 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:30.944227 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2221) took 1043 ms. Trace:
I20260709 12:18:30.878460 22819 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.948807 22825 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.949061 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.946609 18367 rpcz_store.cc:276] 0709 12:18:29.901273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.901396 (+   123us) service_pool.cc:224] Handling call
0709 12:18:29.918328 (+ 16932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.923091 (+  4763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.923139 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:29.954601 (+ 31462us) log.cc:844] Serialized 26 byte log entry
0709 12:18:30.411817 (+457216us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.482042 (+ 70225us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.482309 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.482413 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.944019 (+461606us) raft_consensus.cc:1685] finished
0709 12:18:30.944040 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.944166 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73072,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":6329,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":33533,"thread_start_us":431503,"threads_started":1,"wal-append.queue_time_us":642988}
I20260709 12:18:30.950582 22470 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.955519 23010 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:30.952890 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:30.958382 22939 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:30.958814 22817 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.962473 22970 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.958838 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:30.963917 23067 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:18:30.964371 23064 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:30.964993 23064 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:30.966043 23064 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.966460 23057 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.968688 23057 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.966552 23080 raft_consensus.cc:1064] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:18:30.964918 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:30.971066 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.972561 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.973524 19040 raft_consensus.cc:2368] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:30.866914 23063 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:30.930994 23079 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:30.977221 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.980955 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.981635 18369 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:30.982583 18374 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:30.980075 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.984455 19044 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:30.985684 19035 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:30.969821 22825 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:30.987722 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:30.958845 22939 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.987830 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:30.990509 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.992063 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:30.993777 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:30.995451 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:30.997388 18447 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:31.000491 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2521) took 1238 ms. Trace:
I20260709 12:18:31.000877 18442 rpcz_store.cc:276] 0709 12:18:29.761729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.761847 (+   118us) service_pool.cc:224] Handling call
0709 12:18:31.000307 (+1238460us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54003561d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:31.000419 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204198144}
I20260709 12:18:31.001294 23063 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.002064 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:31.002607 18442 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:31.002707 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.003758 19036 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:31.005563 22819 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.007298 22807 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:31.007876 22679 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.008060 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.008471 22929 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.008646 22655 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:30.987679 23055 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:31.009742 23055 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:31.007582 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:31.007596 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.021301 18444 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:30.987424 22974 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.024067 22974 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:30.994993 22825 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.024801 22655 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:31.028110 23006 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.007663 23075 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:31.028964 23075 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:31.028792 23006 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.030280 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.029958 23075 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.028172 22825 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.031571 23052 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:31.032027 23052 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.032423 23052 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:31.032702 22907 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.035765 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2528) took 1095 ms. Trace:
I20260709 12:18:31.032914 19038 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:31.036839 23004 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:30.995532 22939 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:31.038270 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2024) took 1299 ms. Trace:
I20260709 12:18:31.039129 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.039831 18444 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:31.040359 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.040830 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.041000 18375 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:31.025532 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2230) took 1134 ms. Trace:
I20260709 12:18:31.038700 18300 rpcz_store.cc:276] 0709 12:18:29.738487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.741105 (+  2618us) service_pool.cc:224] Handling call
0709 12:18:29.741454 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.802085 (+ 60631us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:29.987377 (+185292us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.907862 (+920485us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b180014ec00. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:18:30.951892 (+ 44030us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.952616 (+   724us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.957376 (+  4760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.957431 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:30.957466 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.038007 (+ 80541us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b540011fb54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:31.038077 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.038205 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9494,"raft.queue_time_us":4410,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":20689,"spinlock_wait_cycles":294204416,"thread_start_us":384843,"threads_started":2,"wal-append.queue_time_us":715263}
I20260709 12:18:31.049924 19035 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:31.051168 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:31.031000 23006 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.007799 22807 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.068993 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.065456 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:31.017555 22970 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:31.036196 18448 rpcz_store.cc:276] 0709 12:18:29.940794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.940957 (+   163us) service_pool.cc:224] Handling call
0709 12:18:29.966233 (+ 25276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.524607 (+558374us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:30.524660 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.558854 (+ 34194us) log.cc:844] Serialized 26 byte log entry
0709 12:18:30.928999 (+370145us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.929066 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.035375 (+106309us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54002594d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:31.035424 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.035481 (+    57us) raft_consensus.cc:1685] finished
0709 12:18:31.035502 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.035707 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":359621,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":22294,"spinlock_wait_cycles":229251968,"thread_start_us":702934,"threads_started":2,"wal-append.queue_time_us":348945}
I20260709 12:18:31.059811 22974 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:31.062071 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:31.019907 22870 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:31.063530 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.064182 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.056725 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.096421 19040 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:31.042230 18443 rpcz_store.cc:276] 0709 12:18:29.890995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.891168 (+   173us) service_pool.cc:224] Handling call
0709 12:18:29.912128 (+ 20960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.917528 (+  5400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.917567 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:29.949045 (+ 31478us) log.cc:844] Serialized 506 byte log entry
0709 12:18:30.656368 (+707323us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:30.656422 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.656510 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.012375 (+355865us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b24001e9de8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dcdfb0 00007f3c68a44488 00007f3c68a44be8 00007f3c68a1c398 00007f3c68a1776a 00007f3c68a16bb6
0709 12:18:31.016117 (+  3742us) raft_consensus.cc:1685] finished
0709 12:18:31.016133 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.025475 (+  9342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32394112,"thread_start_us":632481,"threads_started":1,"wal-append.queue_time_us":818808}
I20260709 12:18:31.096632 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.010666 23055 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.087709 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.087831 22787 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:30.996886 22831 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:18:31.105041 19038 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:18:31.105826 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 43825.9us
I20260709 12:18:31.097536 19043 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:31.106742 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.108382 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:31.109246 18373 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:31.024269 23080 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.067535 23079 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:31.112046 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.073696 22807 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:31.072525 23004 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.113828 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57162.7us
W20260709 12:18:31.124804 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.117304 18366 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:18:31.086660 23006 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:31.078805 22970 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.095695 22870 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.133719 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.136843 18441 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:31.105394 22787 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.138444 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:31.138983 22957 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.139670 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.142321 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.119454 23079 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.144169 18366 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:31.140457 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.114879 22974 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.157188 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:31.160532 19044 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:31.156032 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:31.161232 18443 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.162091 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.161931 23078 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:18:31.162676 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:31.163429 23076 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.166893 18440 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:31.140064 22787 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.127512 22694 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:31.148252 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.173679 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.136548 22970 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.187108 19043 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:31.140398 22870 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.116457 22947 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.136399 22770 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:31.179177 22858 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.217221 18442 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:31.160219 22974 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.144037 23058 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.159807 23079 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.132800 22862 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:18:31.133805 23004 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.239169 18373 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:31.251201 23084 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 12:18:31.254789 22907 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.187193 22957 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.169961 22644 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:18:31.227083 22694 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:31.314721 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.236384 23072 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:18:31.339816 19036 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:31.347337 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:31.341758 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:31.350485 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 216250us
I20260709 12:18:31.243468 22974 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.341306 23085 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.360551 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.262579 22858 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:31.355976 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 230832us
I20260709 12:18:31.361799 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:31.361965 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.346735 22819 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.362573 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.363655 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.364552 18296 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:31.346681 18373 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:31.368018 18263 leader_election.cc:400] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:31.368369 18263 leader_election.cc:403] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:31.370774 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.372630 18300 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:31.374284 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:31.374573 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.375586 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.375874 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.377112 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.380056 22947 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.380677 23081 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.381217 23081 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.383512 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.384207 23004 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.384276 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.362641 22962 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:31.385141 22962 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.385507 22962 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.387261 23078 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:31.387737 23078 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.388137 23078 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:31.390326 22787 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.392261 22829 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:31.392737 22829 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:31.393623 23088 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:18:31.394876 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2228) took 1521 ms. Trace:
W20260709 12:18:31.395390 23089 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:31.395321 18445 rpcz_store.cc:276] 0709 12:18:29.873188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.887526 (+ 14338us) service_pool.cc:224] Handling call
0709 12:18:29.898798 (+ 11272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.918071 (+1019273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.918114 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.947458 (+ 29344us) log.cc:844] Serialized 26 byte log entry
0709 12:18:31.016093 (+ 68635us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:31.016124 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.016182 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.394600 (+378418us) raft_consensus.cc:1685] finished
0709 12:18:31.394627 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.394807 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6253,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4884,"thread_start_us":1067494,"threads_started":2,"wal-append.queue_time_us":184864}
I20260709 12:18:31.278504 23058 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.404211 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.333211 22907 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.404872 19043 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:31.406098 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.406597 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.399112 19038 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:31.401803 18375 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:18:31.407367 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0557437710001523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:18:31.408620 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0143628299999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:18:31.412020 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.412214 18299 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:31.409011 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0062437710000722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:18:31.415760 23093 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:31.415881 18298 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:31.417184 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.419294 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.419901 23006 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:31.423754 22831 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.424396 22831 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.424819 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.423547 23081 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.425400 18300 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:31.426849 22831 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.426990 22858 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.423492 23096 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:31.358382 22916 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.428731 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.433007 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.467074 18293 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:31.444885 22829 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.446333 23057 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:31.450153 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.472744 18447 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:31.450814 18442 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:31.451972 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.453159 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:31.454994 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.464437 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.469417 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.470459 18296 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:31.358244 22957 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.483812 19040 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.497306 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:31.494482 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.474880 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:31.513029 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 103575us
I20260709 12:18:31.513639 18300 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:31.469985 23063 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.370069 23085 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.466253 23004 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:31.517731 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:31.397094 22819 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:31.523691 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.405501 22825 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.472486 23084 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:31.528745 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:31.378726 22694 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.472297 23057 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:31.503065 23065 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:31.472440 23064 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:31.517347 22829 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:31.481709 22817 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:18:31.472528 22916 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.530520 23069 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.540000 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.570772 18367 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:31.570677 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 101510us
I20260709 12:18:31.627835 19043 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:31.534827 23004 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:31.580927 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 174391us
I20260709 12:18:31.534777 23063 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.588429 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:31.581885 22939 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.012s
I20260709 12:18:31.628425 19040 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:31.629755 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:31.632082 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 117651us
W20260709 12:18:31.637574 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 173220us
I20260709 12:18:31.639989 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:31.640863 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51117.6us
I20260709 12:18:31.584400 22929 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.589164 22914 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.626686 22774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43588 (request call id 19) took 2358 ms. Trace:
I20260709 12:18:31.668069 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:31.647625 22679 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.002s
I20260709 12:18:31.631805 22945 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:31.574306 22831 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:18:31.692313 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.692664 22844 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:31.694147 23058 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:31.649968 18263 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea246fa8cc7e4edbb136da2ee92fb1d3. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:31.650270 23094 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:18:31.650892 22819 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.638506 23069 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.651890 23085 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:31.653046 22916 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.654583 22817 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:31.655504 22907 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.633819 23004 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:31.657708 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:31.655032 18369 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.530s	user 0.019s	sys 0.011s
I20260709 12:18:31.641413 22939 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:31.650321 23103 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:18:31.665225 23057 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.676681 23108 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.677680 23096 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:31.681159 22787 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.682814 23063 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:31.677826 22807 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.013s
I20260709 12:18:31.684433 23100 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:18:31.687541 18367 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:18:31.688369 22962 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.729995 22962 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.731330 23100 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:31.731781 23100 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:31.732686 23100 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.733958 23100 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.690008 23006 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:31.739501 23079 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.740922 22870 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.741901 23079 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.743935 18374 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:31.742976 22870 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:31.690174 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 296) took 22480 ms (client timeout 15000 ms). Trace:
I20260709 12:18:31.665280 22929 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:31.698514 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 60306us
I20260709 12:18:31.753788 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.702173 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.755918 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:31.704918 18369 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:31.671195 23055 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.707855 18296 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:31.758426 23055 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.758783 23057 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.760390 23055 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:31.707775 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:31.709847 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 68322.2us
I20260709 12:18:31.707719 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.761286 18292 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:31.761381 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:31.762353 18299 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:31.763413 22679 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:31.764472 23069 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:31.718169 18443 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.545s	user 0.022s	sys 0.014s
I20260709 12:18:31.699620 22949 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.723397 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.766285 18293 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.701498 22644 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.699817 22945 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:31.721134 23108 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.727031 23111 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.727349 22807 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.727429 18198 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:18:31.760803 23052 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.008s
I20260709 12:18:31.761276 23006 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:31.766888 18443 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:31.769025 18298 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:31.647744 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.793372 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:31.793897 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146069us
I20260709 12:18:31.787431 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.787557 18370 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:18:31.752985 22970 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
I20260709 12:18:31.789573 18366 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:31.790902 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 316064us
I20260709 12:18:31.803503 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:31.804257 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.805516 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:31.791265 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 264622us
I20260709 12:18:31.793985 18367 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:31.806263 18292 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:31.779173 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 293628us
I20260709 12:18:31.796684 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.807159 18369 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:31.797446 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:31.808574 18372 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.812186 18299 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:31.760390 22929 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:31.765278 22825 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:31.776808 22644 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.824388 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.824851 23080 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.826189 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:31.828676 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:31.831939 23109 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:31.833200 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:31.879935 18443 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:31.813377 22962 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.879477 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 60923us
I20260709 12:18:31.837318 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:31.840638 23108 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:31.854279 18293 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:31.786507 23055 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:31.876210 18372 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:31.816915 18198 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:31.880288 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.881829 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:31.888700 19043 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:31.792431 22862 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:31.799664 22870 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.876289 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:31.890489 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64356.5us
I20260709 12:18:31.897676 19038 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:31.890731 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66430.3us
I20260709 12:18:31.796244 23098 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.881011 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:31.780395 22949 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:31.893005 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64407.4us
I20260709 12:18:31.895678 18293 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:31.900460 18372 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:31.902243 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.904307 18442 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:31.880812 22644 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.901172 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.794342 22974 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:31.885509 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56980.3us
I20260709 12:18:31.909960 18198 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 5 to 7, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 7 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:31.914278 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.662688 22774 rpcz_store.cc:276] 0709 12:18:29.225164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.225373 (+   209us) service_pool.cc:224] Handling call
0709 12:18:29.295313 (+ 69940us) log.cc:844] Serialized 506 byte log entry
0709 12:18:29.577031 (+281718us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b1800174ee0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68dd5b0b 00007f3c68e5ab63 00007f3c68e82752 00007f3c6de42d29
0709 12:18:29.609656 (+ 32625us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b1800174ee0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:31.579454 (+1969798us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79121,"raft.queue_time_us":134720,"raft.run_cpu_time_us":2184,"raft.run_wall_time_us":88891,"spinlock_wait_cycles":14856832,"thread_start_us":153361,"threads_started":1,"wal-append.queue_time_us":276234,"wal-append.run_cpu_time_us":1819,"wal-append.run_wall_time_us":1006945}
I20260709 12:18:31.794198 23078 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.948824 18292 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:18:31.949275 18292 pending_rounds.cc:85] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 7
I20260709 12:18:31.949672 18292 pending_rounds.cc:107] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:18:31.892252 23110 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:31.953867 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52799.7us
I20260709 12:18:31.921134 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:31.954737 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:31.899484 23108 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.894598 22694 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.891422 23100 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:18:31.920043 18198 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:31.946947 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:31.814062 22970 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.898554 22870 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.828698 23079 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:18:31.907306 22251 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:31.904011 23098 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.949241 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:31.949874 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.893057 23084 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.949899 18292 raft_consensus.cc:2889] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:31.916241 23076 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.752s	user 0.004s	sys 0.043s
I20260709 12:18:31.956343 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:32.090627 18296 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:32.090987 18291 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:18:31.899101 23069 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:18:31.984089 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:31.984510 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 63100.2us
I20260709 12:18:31.909461 23009 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:18:31.957087 22694 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.794489 23111 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.896188 22962 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:31.985857 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:31.897012 23055 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:32.054562 18298 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:31.953817 22655 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:31.983520 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:31.946251 23109 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:31.957158 22945 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:31.981988 22770 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.067044 18198 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:31.798358 23052 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.058614 23112 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:32.112679 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 128615us
I20260709 12:18:32.119715 22819 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:32.112957 18375 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:32.114156 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 167325us
I20260709 12:18:32.115844 23100 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:32.115752 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 160336us
I20260709 12:18:31.982399 23108 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:31.747838 18196 rpcz_store.cc:269] 0709 12:18:09.210189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.210420 (+   231us) service_pool.cc:224] Handling call
0709 12:18:31.689894 (+22479474us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:25.823067 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:25.935900 (+112833us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:25.938026 (+  2126us) write_op.cc:435] Acquired schema lock
0709 12:18:25.949964 (+ 11938us) tablet.cc:662] Decoding operations
0709 12:18:26.060361 (+110397us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:26.060426 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:26.060463 (+    37us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:18:26.061569 (+  1106us) tablet.cc:701] Row locks acquired
0709 12:18:26.061584 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:18:26.061742 (+   158us) write_op.cc:270] Start()
0709 12:18:26.061861 (+   119us) write_op.cc:276] Timestamp: P: 1783599506061692 usec, L: 0
0709 12:18:26.061880 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:18:26.107972 (+ 46092us) log.cc:844] Serialized 4790 byte log entry
0709 12:18:26.519421 (+411449us) op_driver.cc:464] REPLICATION: finished
0709 12:18:26.845049 (+325628us) write_op.cc:301] APPLY: starting
0709 12:18:26.849494 (+  4445us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:27.110106 (+260612us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:27.134288 (+ 24182us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:28.039107 (+904819us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:28.115983 (+ 76876us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:18:28.126521 (+ 10538us) write_op.cc:312] APPLY: finished
0709 12:18:28.195610 (+ 69089us) log.cc:844] Serialized 113 byte log entry
0709 12:18:29.180301 (+984691us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c65d720ac 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b
0709 12:18:29.578454 (+398153us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:29.578683 (+   229us) write_op.cc:454] Released schema lock
0709 12:18:29.578825 (+   142us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:29.657444 (+ 78619us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:29.657476 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":61478,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":228251,"apply.run_cpu_time_us":59505,"apply.run_wall_time_us":2809503,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54239,"num_ops":12,"prepare.queue_time_us":325176,"prepare.run_cpu_time_us":33681,"prepare.run_wall_time_us":439168,"raft.queue_time_us":257299,"raft.run_cpu_time_us":15265,"raft.run_wall_time_us":158168,"replication_time_us":441041,"spinlock_wait_cycles":58780928,"thread_start_us":1767267,"threads_started":5,"wal-append.queue_time_us":1182730,"wal-append.run_cpu_time_us":6743,"wal-append.run_wall_time_us":2057318}]]}
I20260709 12:18:32.082818 22870 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.139539 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 157309us
I20260709 12:18:32.104182 23098 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.980032 22914 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.147709 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:32.109557 22831 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:32.136575 18294 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:32.148371 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:32.158331 18198 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 6, term changed from 5 to 10, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66), NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 10 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:32.173491 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 226816us
I20260709 12:18:32.108150 23076 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.141726 22251 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:32.117205 22939 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.115367 22694 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.169291 23124 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:18:32.158588 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.110188 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:32.160815 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:32.157531 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:32.161335 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:32.147670 23006 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.178399 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68307.6us
I20260709 12:18:32.170653 18444 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:32.156872 23112 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:32.111761 22947 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:32.176192 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:32.116550 23084 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.152002 22655 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.183326 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:32.187657 18198 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:32.178730 23006 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:32.189211 18293 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.192190 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:32.193784 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:32.194092 18375 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:32.197170 18198 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 8 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.175441 22831 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:32.168469 23100 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:32.194778 23082 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:32.203326 18198 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 9, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 9 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:32.189251 23081 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:32.189879 23112 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.221781 19037 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:18:32.191020 22819 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.209448 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.211957 18293 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:32.212639 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:32.204452 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:32.215780 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:32.216017 23115 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:32.216948 23121 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.263s	user 0.001s	sys 0.001s
I20260709 12:18:32.195277 22974 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.199704 22957 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.191156 23110 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:18:32.218011 22949 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:32.201467 23109 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:32.258700 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 304014us
I20260709 12:18:32.216302 23076 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:18:32.259560 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.216368 23122 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.199379 22939 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.250057 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.198941 23085 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.223285 23120 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:18:32.217619 23127 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:32.254594 18198 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:32.257931 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.231771 19038 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:32.220379 23078 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:32.269176 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 56634.3us
I20260709 12:18:32.270275 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.213416 23081 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:32.220541 23079 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:18:32.254289 23058 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:32.275084 22858 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:18:32.267336 22819 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:32.276069 18448 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:32.391161 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.275391 18447 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:32.275166 22939 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:32.403537 18198 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:32.395305 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.257019 23133 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:32.397374 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:32.272722 23115 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:32.402570 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 218333us
I20260709 12:18:32.425305 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.426736 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.272730 23122 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:32.416067 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.401712 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.406091 18445 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:32.271931 22949 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.272891 22970 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.392861 18442 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:32.298691 23085 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.411110 18198 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 1 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.315826 22957 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:18:32.392273 22470 raft_consensus.cc:2764] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:18:32.421669 18339 leader_election.cc:336] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.560s (SENT)
I20260709 12:18:32.450047 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:32.404856 23057 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.646s	user 0.005s	sys 0.004s
I20260709 12:18:32.428790 18449 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:32.391161 23081 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:32.438184 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.431608 22949 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.439514 23137 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.440753 23085 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:32.441150 19043 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:32.442567 23139 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:18:32.434957 23064 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.442919 23134 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:32.443305 23125 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 12:18:32.443598 23123 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:18:32.436427 18198 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 10, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.403162 23078 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:32.418036 23120 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:32.406410 22939 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.541069 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1089) took 1091 ms. Trace:
I20260709 12:18:32.541407 18441 rpcz_store.cc:276] 0709 12:18:31.450044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.451240 (+  1196us) service_pool.cc:224] Handling call
0709 12:18:31.451731 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:31.520088 (+ 68357us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:31.520511 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.786122 (+265611us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:31.806635 (+ 20513us) log.cc:844] Serialized 52 byte log entry
0709 12:18:31.806967 (+   332us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:31.807510 (+   543us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.847755 (+ 40245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.847857 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.540860 (+693003us) raft_consensus.cc:1685] finished
0709 12:18:32.540877 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.541018 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52793,"prepare.queue_time_us":10836,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":101,"raft.queue_time_us":152354,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":17499,"thread_start_us":319109,"threads_started":2,"wal-append.queue_time_us":540399}
I20260709 12:18:32.544621 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.545706 23100 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:32.548480 18339 leader_election.cc:336] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.671s (SENT)
W20260709 12:18:32.550738 18339 leader_election.cc:336] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.674s (SENT)
I20260709 12:18:32.551174 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.556077 18198 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.557916 18198 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.564834 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.565567 18441 raft_consensus.cc:2376] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15. Re-sending same reply.
I20260709 12:18:32.566629 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.573110 19043 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:32.407611 23133 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.439116 23109 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 12:18:32.457888 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 274644us
I20260709 12:18:32.614007 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.428112 23108 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.431398 23122 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.465008 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:32.574050 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 303847us
I20260709 12:18:32.465464 18448 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:32.637122 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.428643 23058 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:32.453909 22970 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.442700 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.293761 22831 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.296164 23107 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.180s
I20260709 12:18:32.472760 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.708168 18445 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:32.492969 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.468370 23057 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.708873 18440 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:18:32.499157 23128 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:18:32.470057 23137 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.471786 23134 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:32.519775 22655 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.458225 22962 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:32.467185 23129 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:32.530613 23004 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.823s	user 0.007s	sys 0.001s
I20260709 12:18:32.472381 23111 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:32.272258 22251 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:32.465366 22949 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.559103 18198 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.465221 23052 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.730878 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:32.649717 18444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1082) took 1631 ms (client timeout 2111 ms). Trace:
I20260709 12:18:32.637753 18441 raft_consensus.cc:2376] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15. Re-sending same reply.
I20260709 12:18:32.651933 18446 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:32.705585 18295 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:32.712462 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.717288 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:32.733178 19038 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:32.718195 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.719430 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.653468 23078 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.722796 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.721338 22831 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.495666 22819 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.743060 18198 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 9, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.707564 23108 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.707341 23085 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.725222 19009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.016s
W20260709 12:18:32.765169 19009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.008s	sys 0.017s
W20260709 12:18:32.765621 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 589447us
W20260709 12:18:32.719137 22870 consensus_meta.cc:287] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 12:18:32.637739 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.737016 23145 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:32.651472 23069 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:32.731719 18444 rpcz_store.cc:269] 0709 12:18:31.017903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.040553 (+ 22650us) service_pool.cc:224] Handling call
0709 12:18:32.649507 (+1608954us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002026d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:32.649647 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596718336}
W20260709 12:18:32.767114 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 129435us
I20260709 12:18:32.738765 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:32.740396 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.740795 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.650431 23133 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:32.742357 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.769471 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.771804 18443 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:32.762826 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.738163 18445 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:32.772739 18448 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.772868 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:32.771107 18441 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:32.770686 23069 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:32.775033 23069 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.776634 23149 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.776432 23085 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.778708 23085 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:32.770478 23108 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.780606 23137 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.780953 23134 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:32.780579 23122 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.782900 23108 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:32.783362 22870 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:32.784134 23145 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.778232 18198 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:32.808826 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:32.811843 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:32.812377 18295 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:32.814334 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1043) took 1013 ms. Trace:
I20260709 12:18:32.816216 18448 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
W20260709 12:18:32.816849 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.817732 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:32.814716 19035 rpcz_store.cc:276] 0709 12:18:31.800516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.800669 (+   153us) service_pool.cc:224] Handling call
0709 12:18:31.835377 (+ 34708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:31.886120 (+ 50743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:31.886162 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:31.900994 (+ 14832us) log.cc:844] Serialized 26 byte log entry
0709 12:18:32.140066 (+239072us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:32.140106 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.140169 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.814143 (+673974us) raft_consensus.cc:1685] finished
0709 12:18:32.814161 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.814278 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113726,"raft.queue_time_us":61149,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":13395,"thread_start_us":192077,"threads_started":1,"wal-append.queue_time_us":266719}
I20260709 12:18:32.821662 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:32.822331 19037 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:32.823935 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.821750 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:32.832538 18448 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:32.833912 18445 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:32.729046 23004 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.737129 23058 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:32.806958 22962 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:32.862993 18198 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:32.860724 23107 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:32.815706 23085 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:32.815327 23137 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:32.925357 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 183154us
I20260709 12:18:32.874992 23119 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:32.926939 18300 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:32.858386 22916 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:32.929385 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.934715 18446 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.925037 23122 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:32.927802 18198 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:32.857718 23022 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.936028 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.823657 23108 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 12:18:32.940539 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.949257 18198 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 9, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 9 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.891467 23151 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.954022 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2434) took 1138 ms. Trace:
I20260709 12:18:32.955377 18292 rpcz_store.cc:276] 0709 12:18:31.816073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.816368 (+   295us) service_pool.cc:224] Handling call
0709 12:18:31.885076 (+ 68708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.109893 (+224817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.953717 (+843824us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b5400001754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:32.953937 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145818368}
I20260709 12:18:32.728513 23115 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:32.925530 23137 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:32.970995 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:32.976037 18443 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:32.973798 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45872.7us
I20260709 12:18:32.933583 22916 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:32.994313 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:32.998401 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:32.998678 18198 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 10, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 10 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.939472 23107 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:32.938022 23085 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:33.008595 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.009753 18443 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:18:32.995484 22644 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.042s	user 0.010s	sys 0.002s
I20260709 12:18:32.934331 22962 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:32.959394 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141750us
I20260709 12:18:32.975574 23137 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.974138 23138 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:33.015270 18198 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 8, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 8 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.970021 23151 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.023418 18446 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:33.023976 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.038537 19038 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:33.025084 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.039170 19043 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.026556 19044 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:33.031104 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.039363 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.036703 22916 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.041538 23145 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.041648 22807 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:33.046880 23081 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.002s
I20260709 12:18:33.047297 22251 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:33.047364 23081 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.049640 23081 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.050024 18198 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:33.051710 18198 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:33.052459 23135 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:33.035737 22644 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.053117 23135 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.054754 22644 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.055218 23135 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.052892 18198 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 12, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 12 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:33.070443 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.072372 19044 raft_consensus.cc:2368] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:32.952332 23022 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.080847 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.011649 22862 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:33.082785 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.047847 22251 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.100351 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.101189 18367 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:33.090577 18375 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:33.073783 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 220359us
I20260709 12:18:33.090407 18198 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 6 to 10, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 10 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:33.103263 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.103470 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.104213 18372 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:33.091738 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.092842 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.105334 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:32.948056 23119 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.064531 23138 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.099721 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.113469 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.106101 18371 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:33.090035 22807 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:33.110250 18198 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:33.112061 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37ef0235d410491ab47198988075f6e4"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:33.119789 19044 raft_consensus.cc:606] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:33.108906 19043 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:33.116905 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.117597 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:33.092878 23076 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.140170 19044 raft_consensus.cc:614] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:33.143591 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.143568 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.141788 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:33.144968 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.103096 22251 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:33.103108 22470 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.153051 18371 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:33.119040 23138 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:33.111937 23081 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:33.157361 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.135128 23129 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.175453 23129 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:33.176399 23098 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.987s	user 0.007s	sys 0.004s
I20260709 12:18:33.176877 23098 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.178226 23129 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.140235 22858 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:18:33.143009 23085 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.144208 23107 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.158921 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.118057 23149 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:33.135908 22870 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.167296 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.167356 22655 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.197382 18374 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.198611 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.199254 23081 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.199851 23132 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.200343 23132 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.201380 23081 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.202382 23132 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.203495 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.204154 18294 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:33.154426 23122 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.169449 22962 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.171285 22819 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:18:33.143532 23109 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:33.147544 23057 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.160359 19044 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:33.192113 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.140482 22807 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.194867 18367 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:33.225399 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.226449 18291 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:33.195093 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.107543 22862 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.167774 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.207697 18292 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:33.199175 18374 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:33.195566 22251 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.164340 22774 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:33.165058 23075 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.222751 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.192734 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.207098 23076 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.163779 22470 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.207737 23004 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.158389 22685 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:18:33.231339 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.213624 23122 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.256294 22862 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:33.241248 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73544.9us
I20260709 12:18:33.267838 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.268980 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.243430 18294 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:33.191636 23135 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:18:33.227769 23145 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:33.226215 23057 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.212893 23149 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.259152 23069 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:33.262205 22939 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.721s	user 0.007s	sys 0.000s
I20260709 12:18:33.262512 23159 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:33.246778 22819 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.239661 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.264700 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.286551 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.286928 18372 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:33.287284 18295 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:33.227957 23150 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 12:18:33.252168 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.273449 22858 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.239998 22962 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:33.266566 22949 consensus_meta.cc:287] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.003s
I20260709 12:18:33.282708 23098 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.265326 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.265766 23122 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.290902 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.283252 23110 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:33.338210 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 47375.3us
I20260709 12:18:33.291584 22251 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.279130 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:33.322881 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 255970us
I20260709 12:18:33.307888 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.328209 18367 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.284545 23052 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:33.265904 22870 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:18:33.346762 18374 raft_consensus.cc:2368] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:18:33.221971 23109 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:33.335944 22970 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
I20260709 12:18:33.314144 23075 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.337713 22945 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.286176 22939 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.366791 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.367551 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.348879 23052 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.349146 22858 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.343529 23129 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:33.343773 23063 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:33.372732 18300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2430) took 1671 ms (client timeout 2026 ms). Trace:
I20260709 12:18:33.383714 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.389523 18442 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:33.390246 18300 rpcz_store.cc:269] 0709 12:18:31.701813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.701929 (+   116us) service_pool.cc:224] Handling call
0709 12:18:33.372540 (+1670611us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540040e554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:33.372663 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2693880704}
I20260709 12:18:33.391440 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.393285 18444 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:33.361637 23110 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.375044 18447 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:33.337929 22949 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.390718 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.394835 18441 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.397858 18367 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
W20260709 12:18:33.399715 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.357121 23151 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.405113 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.362119 22970 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.379385 23134 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.416355 18368 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:33.418004 23055 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:33.419294 22694 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.227s	user 0.010s	sys 0.002s
I20260709 12:18:33.393187 23063 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.388494 23052 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:33.452648 23112 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.198s	user 0.008s	sys 0.002s
I20260709 12:18:33.453053 23112 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.425391 18372 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.457265 18440 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:33.371866 22819 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:18:33.428834 23109 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.395983 23129 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.463752 23162 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:33.464172 23162 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:33.464387 22807 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.404531 22962 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.465135 22962 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:33.465269 23129 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.466464 23069 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.467562 23057 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.424206 23119 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:33.468529 23133 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
I20260709 12:18:33.469168 23063 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.468976 23133 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.469695 22694 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:33.430888 23176 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:18:33.474493 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:33.475374 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.475366 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 196481us
I20260709 12:18:33.434393 18441 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:33.477283 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:33.434619 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.436900 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2442) took 1262 ms. Trace:
I20260709 12:18:33.478734 18369 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:33.478796 19035 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:33.480371 18449 raft_consensus.cc:2368] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:33.480903 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.481446 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.439756 22655 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:33.485441 18412 leader_election.cc:400] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:33.485796 18412 leader_election.cc:403] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.487679 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.443848 23138 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.444576 22949 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.444983 18298 raft_consensus.cc:2368] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12.
W20260709 12:18:33.445199 23171 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 12:18:33.445719 23084 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.447996 23134 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.448778 22916 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.379920 23161 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:18:33.450451 23149 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.464881 23162 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.470785 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.431785 22470 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.471392 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.471788 18444 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:33.477932 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.517202 18441 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:33.488885 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.478523 18299 rpcz_store.cc:276] 0709 12:18:32.174069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.174220 (+   151us) service_pool.cc:224] Handling call
0709 12:18:32.174501 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.224298 (+ 49797us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:32.256265 (+ 31967us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.232014 (+975749us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.241375 (+  9361us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.241446 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:33.241507 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.436697 (+195190us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:33.436737 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.436846 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8323,"prepare.queue_time_us":407024,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":5181,"raft.queue_time_us":49442,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":14563,"spinlock_wait_cycles":113869056,"thread_start_us":966139,"threads_started":3,"wal-append.queue_time_us":601587}
I20260709 12:18:33.471310 22787 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:33.490303 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2540) took 1028 ms. Trace:
I20260709 12:18:33.490710 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.455143 23112 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.526515 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.527760 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.528509 19040 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:33.490869 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.491016 18443 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.494378 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.530319 18446 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:33.530769 18294 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:33.495682 18291 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:33.495747 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:33.496411 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.497020 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.498359 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.499087 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.503072 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.471930 23108 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:33.506739 23133 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.483361 22945 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.483584 22679 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:33.508775 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.509099 23022 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.494604 23137 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.496212 23063 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:33.494019 23098 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:33.512527 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.515558 18447 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:33.522409 18440 rpcz_store.cc:276] 0709 12:18:32.461823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.721770 (+259947us) service_pool.cc:224] Handling call
0709 12:18:33.490105 (+768335us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400202bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:33.490251 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668517760}
I20260709 12:18:33.523669 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.501641 23109 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 8. Current term is 9.
I20260709 12:18:33.494817 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.533705 18297 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:33.527815 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.551349 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.555954 19036 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.638854 18292 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:33.546144 22251 consensus_queue.cc:1175] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" responder_term: 10 status { last_received { term: 10 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 9 } }
W20260709 12:18:33.557886 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60992.8us
I20260709 12:18:33.652331 19040 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:33.514026 23161 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:33.620916 18444 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:33.500447 23057 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 12:18:33.632519 18449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2536) took 1171 ms (client timeout 1560 ms). Trace:
I20260709 12:18:33.556664 23168 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:18:33.557909 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 59587.4us
I20260709 12:18:33.637771 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.557039 22685 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:33.647543 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:33.641819 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 114109us
W20260709 12:18:33.689006 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 51320us
I20260709 12:18:33.689489 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.689694 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:33.635110 23098 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:33.484035 22655 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:33.694233 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.694977 18299 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:33.696538 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.697120 18299 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:18:33.697252 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.644068 18374 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:33.545701 22787 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.562630 23129 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:33.557024 23135 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:18:33.557799 23108 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.619859 23162 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.631732 22679 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:33.641803 23063 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:33.685827 18449 rpcz_store.cc:269] 0709 12:18:32.460749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.474340 (+ 13591us) service_pool.cc:224] Handling call
0709 12:18:33.628963 (+1154623us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:33.632452 (+  3489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131977088}
I20260709 12:18:33.679113 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.686285 18292 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:33.646183 22770 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:33.698921 22957 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.703006 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.700989 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.698387 18374 pending_rounds.cc:85] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:33.718359 18374 pending_rounds.cc:107] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:18:33.718638 18374 raft_consensus.cc:2889] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:33.704151 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.704607 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.720410 19040 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:33.706712 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.693477 23168 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.708444 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.722671 18298 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:33.708863 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.687620 23006 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.710165 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.705705 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.694906 22914 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:33.695513 23151 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:33.691861 23084 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.716760 18291 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:33.701298 23098 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.688045 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:33.738322 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 50398.2us
I20260709 12:18:33.725087 18295 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:33.725368 18297 raft_consensus.cc:2368] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:18:33.722174 23081 raft_consensus.cc:2764] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:33.711858 22694 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.752077 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.752807 23137 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.754222 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:33.786046 19041 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.754817 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:33.758306 23178 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.425s	user 0.000s	sys 0.007s
W20260709 12:18:33.762492 23173 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 12:18:33.763438 23145 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.766780 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.771106 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.730592 23112 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:33.776292 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.725087 23168 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:33.778565 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.773536 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48802.2us
I20260709 12:18:33.777107 19037 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:33.727466 22914 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.822146 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.727695 23109 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.796446 19039 raft_consensus.cc:2368] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:18:33.798540 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.800062 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.734298 23151 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.730705 22770 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:33.802151 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.771811 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.834214 18298 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:33.732668 22685 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:33.808967 19044 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:33.778827 22819 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:33.823804 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.724681 22957 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.829346 18366 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:33.830135 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:33.868310 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:33.805742 23188 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:18:33.835858 19006 leader_election.cc:400] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12.
I20260709 12:18:33.871742 19006 leader_election.cc:403] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.838088 22787 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:33.839051 22770 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:33.839815 22945 raft_consensus.cc:2764] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:18:33.820799 23052 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.844291 19039 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:33.858722 23063 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.859465 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.860441 18298 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:33.862159 18295 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:33.889153 18369 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.889674 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.889976 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.891111 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.891958 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.864523 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:33.858822 19041 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:33.892719 18374 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:33.864708 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:33.809144 23145 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:33.869761 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.776371 22470 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:18:33.873775 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:33.875119 22251 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:33.883728 23189 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:18:33.826881 22914 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:33.886741 18366 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:33.830976 23138 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.769276 22655 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.834851 23151 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:33.797237 23181 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:18:33.875746 22644 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:18:33.899394 22251 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.876506 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:33.893180 23185 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:18:33.922209 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.877465 23100 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.904070 18448 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:33.902638 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.906881 18369 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:33.892122 23135 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:33.908582 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 187321us
I20260709 12:18:33.834729 23109 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.883950 22819 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:33.887569 23134 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.835588 23112 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:33.896785 23163 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12.
I20260709 12:18:33.895466 19037 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:18:33.916859 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.919663 18266 leader_election.cc:400] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:18:33.922892 18298 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:33.922947 18374 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:33.935281 22251 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:33.917347 22907 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:33.939524 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63100.3us
I20260709 12:18:33.942914 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:33.940701 18266 leader_election.cc:403] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.923098 22655 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:33.943704 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 200865us
W20260709 12:18:33.937119 23190 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:18:33.948019 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:33.913384 23069 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:18:33.948072 23192 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:18:33.964078 18338 leader_election.cc:400] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:18:33.930323 23135 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.950325 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.956477 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:33.958117 18440 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [10.7-14.9]
I20260709 12:18:33.964107 18442 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.951898 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:33.949698 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:33.974216 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:33.957024 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.936908 23159 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:33.969996 18338 leader_election.cc:403] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:18:33.969969 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:33.971817 18447 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:33.942978 23108 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:33.979694 18298 raft_consensus.cc:2376] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15. Re-sending same reply.
I20260709 12:18:33.942591 22907 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:33.945952 22679 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:34.020102 23165 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.682s	user 0.000s	sys 0.008s
I20260709 12:18:34.020417 23196 ts_tablet_manager.cc:933] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:18:33.978863 23159 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:34.021102 23201 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.022289 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.957268 23194 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.957664 22774 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:34.023712 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:34.002439 19041 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:33.934710 23100 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.033703 18442 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:33.981304 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:33.963349 23081 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.054253 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:33.980695 22929 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.046049 18294 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.978022 23135 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.955494 23182 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:18:34.037326 22644 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:34.055259 22819 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:34.058192 22962 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.593s	user 0.018s	sys 0.000s
I20260709 12:18:34.047145 22774 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.036367 23181 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:34.085016 18446 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:34.059981 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.085453 18444 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:34.060228 23100 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.060791 22957 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.061378 23119 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:34.044979 23122 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.001s
I20260709 12:18:34.061722 23149 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:34.061801 18370 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:34.062074 23191 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:34.062590 18295 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:34.039175 22916 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.005s
I20260709 12:18:34.082635 23052 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:34.047966 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66738.6us
W20260709 12:18:34.066190 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92254.7us
I20260709 12:18:34.064040 23109 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.047550 23159 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.036437 22907 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.047240 23194 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.071903 18369 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:34.037647 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:34.141568 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.104446 18444 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: [10.5-11.6]
W20260709 12:18:34.075501 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 125888us
I20260709 12:18:34.079152 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.065295 23201 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:34.022199 23134 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:34.150758 23134 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.036880 23161 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:18:34.082737 19036 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:34.082209 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 116296us
I20260709 12:18:33.948279 22685 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.087639 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.156673 18442 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:34.087007 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.090242 22774 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.124684 18293 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:34.125243 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:34.142958 18441 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:34.143110 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.144011 23206 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:34.144053 23201 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.150017 23109 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:34.155488 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.088241 22962 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.176324 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.177235 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.181638 18442 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:34.181739 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.182111 23109 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:34.183311 23203 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:18:34.124785 23149 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:34.175012 23137 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:34.185328 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.180100 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:34.189116 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 46092.7us
I20260709 12:18:34.189574 23196 tablet_copy_client.cc:323] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:18:34.189922 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.190100 23145 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.192242 23145 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.180225 23052 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:34.184780 23149 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.151122 23134 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.128079 23191 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.151372 23194 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.195140 18263 leader_election.cc:400] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:18:34.201826 23138 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.204365 23191 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.181061 23156 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:34.206924 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:34.092224 23122 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.206239 18296 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:34.186039 23109 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.194253 22929 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.194309 23161 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:34.211971 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:34.124874 22914 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.216593 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:34.206310 23004 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.178498 22644 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:34.207813 23129 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:34.244076 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.245146 22957 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:34.245471 22957 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:34.245767 22957 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.217223 23052 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.248488 18292 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:18:34.209610 23098 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.224817 22685 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:34.234073 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 108902us
I20260709 12:18:34.250844 23211 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.254637 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.254449 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2266) took 1046 ms. Trace:
I20260709 12:18:34.257123 18367 rpcz_store.cc:276] 0709 12:18:33.208489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.226133 (+ 17644us) service_pool.cc:224] Handling call
0709 12:18:34.254297 (+1028164us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b5400395b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:34.254386 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95035648}
I20260709 12:18:34.259001 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:34.243942 22770 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.225013 23078 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:34.243584 23161 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:34.249531 23052 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.266417 23084 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:34.270226 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:34.255613 22685 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.271984 18298 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:34.276234 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:18:34.276871 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet ea246fa8cc7e4edbb136da2ee92fb1d3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3
I20260709 12:18:34.278105 23098 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.281612 23081 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:34.283599 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:34.286056 18310 tablet_copy_source_session.cc:215] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:18:34.292723 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:34.294293 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.270362 23052 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.240525 23134 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.325881 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:34.326627 18298 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:34.283869 22685 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.330606 23205 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:34.296041 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.286944 22770 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.348330 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:34.349001 18298 raft_consensus.cc:2376] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12. Re-sending same reply.
I20260709 12:18:34.349095 22939 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.329823 23129 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.295446 23159 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.327988 22957 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.351635 23129 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.353202 23159 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.353331 22957 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.271437 22907 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.353994 23129 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.354691 23161 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.356565 18295 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:34.358253 23122 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:34.360854 23168 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.000s
I20260709 12:18:34.361243 23168 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.362304 19042 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:18:34.363818 23135 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.365746 22655 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.366387 18291 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:34.366236 22655 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.367717 18291 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [10.7-14.9]
I20260709 12:18:34.368932 18291 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.367739 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.370468 18441 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.351040 18298 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [11.9-12.10]
W20260709 12:18:34.375540 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 120965us
I20260709 12:18:34.376652 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.295964 23081 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:34.382573 18291 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:34.383105 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:34.386461 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 142443us
W20260709 12:18:34.369227 23189 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.444s	user 0.001s	sys 0.000s
I20260709 12:18:34.415122 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.420866 18441 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:34.374457 23134 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.365777 23135 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.429899 18298 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:34.374564 22939 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:34.432006 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.380505 23168 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:34.413654 22957 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.436527 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.442484 18294 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:34.420225 23078 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:34.458545 18371 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:34.460739 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.437121 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.383275 23081 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:34.441949 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.464504 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:34.462401 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:34.462343 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:34.466814 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.346879 23205 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:34.415724 23151 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.471922 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.470587 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.442170 22907 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.462833 18441 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:34.443485 23084 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:34.466332 23081 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.463512 23182 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.475445 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.461184 23215 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:34.485671 19036 raft_consensus.cc:2368] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13.
I20260709 12:18:34.470052 23129 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.495944 23217 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:34.496598 23084 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.497092 23217 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.499178 23220 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:18:34.501181 23205 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:34.505203 23194 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.466022 23057 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:18:34.512780 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:34.515367 23168 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.516299 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.519153 23119 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:34.465659 22957 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.516330 19043 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:18:34.530103 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69420.2us
I20260709 12:18:34.523156 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.462960 23211 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:34.524853 19039 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:34.530849 19044 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:34.530824 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.460480 22770 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.537101 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.532035 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.494033 22914 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.541882 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:34.535609 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 70025.4us
I20260709 12:18:34.476806 23078 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:18:34.537719 18444 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:34.537074 22929 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:34.540009 22957 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.530599 23119 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.524710 23129 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:34.561914 22957 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:34.562059 23215 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.532887 22655 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:34.542883 23109 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.571802 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.549392 23182 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:34.529441 23168 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.551563 22929 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:34.565665 23215 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:34.591534 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.592136 18444 raft_consensus.cc:2376] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13. Re-sending same reply.
I20260709 12:18:34.592831 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.593413 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.551789 23117 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:34.596624 18369 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:34.596979 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.618991 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.620496 23151 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.586498 22929 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:34.629653 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.575162 23227 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:18:34.630947 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:34.631022 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 78451.4us
I20260709 12:18:34.634425 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:34.639834 18444 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:34.640655 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.641925 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.625133 22957 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:18:34.632867 23129 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.590574 23211 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.643200 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:34.587478 23149 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:18:34.659857 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 86205us
I20260709 12:18:34.672721 18444 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:34.675323 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.679018 18369 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:34.680927 18444 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:34.683048 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.684295 18366 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.685247 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.686100 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.685498 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.688711 18371 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.652400 23109 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.671569 23149 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:34.701366 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 808) took 1669 ms. Trace:
I20260709 12:18:34.704103 19035 rpcz_store.cc:276] 0709 12:18:33.023877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.023993 (+   116us) service_pool.cc:224] Handling call
0709 12:18:33.024311 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.479438 (+455127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.692937 (+1213499us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54006be4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:34.693091 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026623360}
I20260709 12:18:34.681795 23181 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:34.651367 23222 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 12:18:34.711565 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.709751 23181 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.687889 23052 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.651623 23221 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 12:18:34.709954 23109 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.715431 23194 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.728348 23231 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:34.718459 23181 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.715170 22655 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.732663 18443 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.739099 23138 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.739565 23138 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.740401 23138 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.715816 22685 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.731523 23231 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:34.737885 22655 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.753419 23168 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:34.728816 23232 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:18:34.753865 18366 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:34.730911 23109 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.754900 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:34.765126 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:34.767042 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.755980 23119 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.758322 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.761880 18371 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:18:34.788687 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.792816 19035 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.797549 23236 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:34.797886 23006 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.798321 23006 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:34.768244 23222 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:34.798902 22655 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.799656 19039 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:34.761619 23138 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.768069 22939 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:34.808801 18375 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.797101 23149 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:34.765249 23117 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:18:34.813349 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.799109 23006 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.815691 19037 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:34.817138 23236 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:34.817523 23236 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:34.820729 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.818406 23236 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.808779 22939 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.834175 18366 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:34.836886 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 877) took 1363 ms. Trace:
I20260709 12:18:34.834998 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:34.840953 18293 rpcz_store.cc:276] 0709 12:18:33.473842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.487335 (+ 13493us) service_pool.cc:224] Handling call
0709 12:18:34.836727 (+1349392us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540021ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:34.836825 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463007104}
W20260709 12:18:34.842971 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 75970.3us
I20260709 12:18:34.844679 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:34.846705 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.831977 23112 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.841799 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:34.810947 22655 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:34.834522 22939 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.836416 23236 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.863051 18444 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:34.837244 22685 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.856334 23111 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.836887 23118 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:18:34.867126 23111 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.867774 23118 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:34.868181 23118 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:34.856742 22831 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:34.857043 23232 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:34.880470 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.859627 22817 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:34.865547 23006 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.886209 22831 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:34.886193 22655 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.883772 23117 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.899017 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:34.908001 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.914690 19042 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:34.915004 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:18:34.915557 18293 raft_consensus.cc:606] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:34.915868 18293 raft_consensus.cc:614] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:34.913712 22655 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.903059 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:34.912618 22831 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.900743 23081 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.884357 23211 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.898500 22817 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:34.922711 18444 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.913769 23117 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.897042 23052 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:34.933485 22817 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.936304 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:34.939103 23246 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:18:34.952435 18445 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:34.952170 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 107557us
I20260709 12:18:34.945158 19043 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:34.946156 23109 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:34.930416 23239 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:18:34.958603 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.966763 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.967399 18371 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:34.885185 23232 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:34.967501 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.967733 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.969115 23255 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.973064 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:34.975303 23253 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:34.975597 19044 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.984885 19044 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:34.986258 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.987983 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:34.988766 19043 raft_consensus.cc:2376] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13. Re-sending same reply.
I20260709 12:18:34.886817 23238 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:18:34.990823 19044 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:34.961647 23119 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:34.951205 22929 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.011819 19042 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:35.017431 18444 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:18:35.018872 23245 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:18:35.020854 22929 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:34.907740 23240 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.253s
I20260709 12:18:35.027768 22655 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:35.030118 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:34.973513 23255 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:34.976063 23253 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:34.968019 23118 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.041076 23118 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.043179 23117 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:35.023940 23119 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:18:35.046326 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:35.046510 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 92753.5us
I20260709 12:18:35.048496 18373 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:35.056025 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:35.078338 23232 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.083133 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:35.088389 22939 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.055850 23081 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:35.094592 23253 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.076829 23181 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.097672 23181 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:35.098563 23214 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.630s	user 0.000s	sys 0.004s
I20260709 12:18:35.103530 22939 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.104774 23255 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.119460 22774 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.131203 23215 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.132820 23215 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.136802 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2276) took 1657 ms. Trace:
I20260709 12:18:35.137171 18372 rpcz_store.cc:276] 0709 12:18:33.479526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.480327 (+   801us) service_pool.cc:224] Handling call
0709 12:18:33.480705 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.723086 (+242381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.723136 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.723200 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:33.723396 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.136593 (+1413197us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54002f8354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:35.136635 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.136749 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556772608}
I20260709 12:18:35.103063 23211 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.145658 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.147174 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.150588 23009 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.140120 23010 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:35.151108 23255 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.157658 22914 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.155830 23009 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.171659 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:35.172675 18444 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.163416 23211 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:35.174141 18372 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:35.180912 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.185549 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:35.186167 19040 raft_consensus.cc:606] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:35.186480 19040 raft_consensus.cc:614] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:35.188874 18444 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:35.190294 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:35.191561 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:35.166039 23010 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.193107 18367 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:35.188470 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:35.197028 18367 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.198412 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:35.200775 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.204577 23133 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.204815 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:35.205108 23133 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.200183 22831 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:18:35.197845 18367 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.172849 22914 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.209969 18368 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:35.211619 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:35.207917 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.206830 22962 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.220780 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.221100 23133 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:35.217058 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.224890 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:35.229182 22251 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:35.229739 22251 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:35.229646 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.233001 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.240854 22251 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:35.244808 23260 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 12:18:35.247032 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:35.247730 23254 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 12:18:35.247812 19040 raft_consensus.cc:606] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.248513 19040 raft_consensus.cc:614] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:35.250095 18370 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:35.256132 18367 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:35.260000 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52209.8us
I20260709 12:18:35.260704 23100 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:35.261232 23100 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.263304 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1056) took 1749 ms. Trace:
I20260709 12:18:35.264415 18367 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:18:35.264917 18367 pending_rounds.cc:85] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 7
I20260709 12:18:35.263751 19038 rpcz_store.cc:276] 0709 12:18:33.513981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.514202 (+   221us) service_pool.cc:224] Handling call
0709 12:18:33.552598 (+ 38396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.716190 (+163592us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:33.727927 (+ 11737us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.887245 (+159318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.887310 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:33.887376 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.263001 (+1375625us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400834fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:35.263043 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.263203 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13551,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":6268,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":5669,"spinlock_wait_cycles":375477504,"thread_start_us":158956,"threads_started":1,"wal-append.queue_time_us":241170,"wal-append.run_cpu_time_us":1217,"wal-append.run_wall_time_us":1050653}
I20260709 12:18:35.265179 18367 pending_rounds.cc:107] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:18:35.265594 18367 raft_consensus.cc:2889] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:35.265739 23135 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:35.265975 18367 pending_rounds.cc:107] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:18:35.266425 18367 raft_consensus.cc:2889] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:35.271919 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.271745 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:35.209260 23084 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:35.284029 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.287396 18293 raft_consensus.cc:2368] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13.
I20260709 12:18:35.283595 23135 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:35.284116 23078 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:35.289510 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.283794 22945 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 12:18:35.290354 23226 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.743s	user 0.002s	sys 0.000s
I20260709 12:18:35.290520 23236 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:35.293128 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:35.293936 19038 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:35.294430 22912 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:35.295161 19040 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 12:18:35.296056 23162 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.296170 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:35.298257 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.298841 18298 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:35.299639 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:35.300016 23135 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:35.300248 19043 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:35.289690 23078 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.301075 23135 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.290721 22945 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:35.303602 18373 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:35.303717 22945 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:35.304642 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:35.304589 22945 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.305624 23215 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:35.305701 22945 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:35.306160 23215 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.306571 23215 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.303282 23078 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:35.307314 23159 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:18:35.309578 18297 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:35.311408 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:35.312268 18368 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:35.314798 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:35.295892 23136 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:35.319728 19043 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:35.320976 19036 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:35.200587 23081 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:18:35.286890 23084 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.288344 23196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea246fa8cc7e4edbb136da2ee92fb1d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:35.345777 23084 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.347186 23181 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:35.348210 23227 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.348634 23227 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:35.349504 23227 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.350896 23227 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.341554 22912 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:35.351845 22912 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.354416 22912 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.355818 23009 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.322510 23085 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:35.365901 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:35.318014 23100 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.320781 23118 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:18:35.324779 23136 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:35.369462 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:35.371817 19036 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.371798 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.376084 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:35.376603 18442 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:35.376890 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.377503 18298 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.378494 23022 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.367568 23085 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.383144 18442 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:35.385111 22251 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.386775 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.389204 18298 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:35.391587 18441 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:35.392617 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.393057 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.385623 22251 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.393841 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.394075 18299 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:35.393038 19036 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:35.392618 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:35.395740 18441 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.395936 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:35.396171 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.397920 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.398419 18298 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:35.398670 18291 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:35.406749 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.381809 23085 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.384881 22914 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.385504 23100 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.416935 18441 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:35.419309 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.421595 18441 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [10.7-14.9]
I20260709 12:18:35.399166 23078 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:35.423463 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:35.424059 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:35.422468 23136 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.426589 23010 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.427188 22251 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.431452 19036 raft_consensus.cc:606] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:35.433784 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.430801 23111 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.408509 23217 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:18:35.454867 18448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1101) took 1994 ms (client timeout 2180 ms). Trace:
I20260709 12:18:35.455511 18449 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:18:35.459756 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:18:35.457078 18448 rpcz_store.cc:269] 0709 12:18:33.460505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.460619 (+   114us) service_pool.cc:224] Handling call
0709 12:18:35.454666 (+1994047us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:35.454810 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891272576}
I20260709 12:18:35.457770 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.437284 23149 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:35.437139 23078 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.431197 23100 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.468925 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.482363 23215 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.468107 23149 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.484568 23149 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.490729 23215 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.494745 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2568) took 1095 ms. Trace:
I20260709 12:18:35.495920 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2452) took 1801 ms. Trace:
I20260709 12:18:35.497820 23231 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.479135 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.501351 18443 rpcz_store.cc:276] 0709 12:18:34.399552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.657592 (+258040us) service_pool.cc:224] Handling call
0709 12:18:34.657856 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.736400 (+ 78544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.494441 (+758041us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b54001a5254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:35.494665 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171389312}
I20260709 12:18:35.502800 18292 rpcz_store.cc:276] 0709 12:18:33.695210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.099704 (+404494us) service_pool.cc:224] Handling call
0709 12:18:34.126071 (+ 26367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.248958 (+122887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.495607 (+1246649us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400274cd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:35.495850 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280899328}
I20260709 12:18:35.511547 18374 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:35.519341 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:35.520169 18373 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:35.524276 18293 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:35.528437 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:35.509140 23267 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:18:35.535876 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:35.512861 23269 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:18:35.539719 22251 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.494925 23084 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.541621 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.516677 23262 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.547372 22251 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.535936 23231 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.536348 22970 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:35.552188 23271 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 12:18:35.562465 22251 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:35.565922 22251 consensus_queue.cc:1243] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:35.567360 22251 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:35.539999 23215 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:35.536785 23222 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:18:35.572712 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.581838 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:35.582506 18293 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:35.583760 18371 pending_rounds.cc:85] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:35.584072 18371 pending_rounds.cc:107] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:18:35.584308 18371 raft_consensus.cc:2889] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:35.565922 23135 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:18:35.574200 23215 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.589257 23215 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.561769 23262 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.589919 22817 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:35.590359 22817 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:35.591212 23262 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.541266 23078 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.594336 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.597875 22817 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.600759 23085 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.603487 23084 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.603953 19042 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:18:35.569757 22970 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.608915 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:35.611054 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.601639 22817 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.612596 23232 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:35.613983 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.616796 22970 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.620782 18445 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:35.622418 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.623389 18443 raft_consensus.cc:2393] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:35.624293 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:35.624852 18368 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:35.613168 23266 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:18:35.639134 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:35.639529 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:35.612882 23080 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:35.598146 23215 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:35.654454 18446 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:35.666551 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.667774 18443 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:35.679092 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.623322 22819 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.690750 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:35.637996 23085 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:35.650642 23205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.106s	user 0.018s	sys 0.008s
I20260709 12:18:35.650942 23129 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:35.641448 23100 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.652225 22694 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:35.707862 18370 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:35.686364 23215 consensus_queue.cc:1243] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:18:35.697580 23205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.153s	user 0.019s	sys 0.008s
I20260709 12:18:35.718609 18368 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:35.657056 23236 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:18:35.692229 23181 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:35.733487 22655 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:35.733999 22655 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:35.737855 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2559) took 1789 ms. Trace:
I20260709 12:18:35.738379 18449 rpcz_store.cc:276] 0709 12:18:33.948790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.948892 (+   102us) service_pool.cc:224] Handling call
0709 12:18:35.455272 (+1506380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.460179 (+  4907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.737611 (+277432us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000d8750. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:35.737782 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3448011520}
I20260709 12:18:35.747749 18447 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:35.724908 22694 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:35.698397 22819 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:35.756691 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 107777us
I20260709 12:18:35.759594 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:35.759758 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2459) took 1889 ms. Trace:
I20260709 12:18:35.760468 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:35.734988 22655 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.769251 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:35.735885 23181 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:35.783841 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.757182 23205 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.760341 18295 rpcz_store.cc:276] 0709 12:18:33.870106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.358239 (+488133us) service_pool.cc:224] Handling call
0709 12:18:34.358521 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.162822 (+804301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.162869 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.165718 (+  2849us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.384846 (+219128us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:35.450123 (+ 65277us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.452581 (+  2458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.452656 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.759473 (+306817us) raft_consensus.cc:1685] finished
0709 12:18:35.759505 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.759674 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796714,"prepare.queue_time_us":40645,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":790129,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":7176,"thread_start_us":282038,"threads_started":3,"wal-append.queue_time_us":294739}
I20260709 12:18:35.791309 23149 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.792323 23149 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.793270 23227 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:35.794183 23227 consensus_queue.cc:1243] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:35.794564 23149 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.795565 23182 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.796201 23182 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.798727 23182 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.804713 22819 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:35.808399 23129 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 6 to 9, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:35.813069 23257 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.735s	user 0.003s	sys 0.000s
I20260709 12:18:35.821051 22655 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:35.824512 22694 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.829080 23221 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:35.829599 23221 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.785845 23080 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:35.833326 23080 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:35.833837 23080 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:35.840075 23280 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:18:35.841717 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 818) took 1327 ms. Trace:
I20260709 12:18:35.842167 19041 rpcz_store.cc:276] 0709 12:18:34.514356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.521643 (+  7287us) service_pool.cc:224] Handling call
0709 12:18:34.522334 (+   691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:34.522795 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:34.522849 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.500915 (+978066us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4c0048dd40. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6711f644 00007f3c67146f2d 00007f3c67146b93 00007f3c68e7718b 00007f3c68e77cb5 00007f3c68e79004 00007f3c68e78729 00007f3c68e7dd96 00007f3c68e72adb
0709 12:18:35.513645 (+ 12730us) log.cc:844] Serialized 564 byte log entry
0709 12:18:35.568080 (+ 54435us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:35.568139 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.568228 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.841330 (+273102us) raft_consensus.cc:1685] finished
0709 12:18:35.841357 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.841643 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11247360,"thread_start_us":51257,"threads_started":1,"wal-append.queue_time_us":197135}
W20260709 12:18:35.844805 23283 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:18:35.847146 18263 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8903d70080d24236be955433a92791c7. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:35.847739 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 552797us
W20260709 12:18:35.855518 23245 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:18:35.857192 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2564) took 1463 ms. Trace:
I20260709 12:18:35.791525 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:35.863816 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.865067 23133 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.865504 23133 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:35.865944 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 74519.7us
I20260709 12:18:35.866822 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.867542 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.868621 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.788059 23252 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.869732 23252 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.870086 19043 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:35.870522 23252 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.871500 23252 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.871721 19038 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:35.873533 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 1.01849e+06us
I20260709 12:18:35.874190 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.869305 19035 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:35.866120 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.876381 19042 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:35.877065 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.877638 19036 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:35.878984 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:35.883010 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.789937 23205 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:35.884281 18371 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:35.884627 23205 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.887362 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:35.889319 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.857689 18440 rpcz_store.cc:276] 0709 12:18:34.393762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.656479 (+262717us) service_pool.cc:224] Handling call
0709 12:18:34.656850 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:34.673905 (+ 17055us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:34.684436 (+ 10531us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.861998 (+177562us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:35.428413 (+566415us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18002dfac0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:35.747540 (+319127us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:18:35.752815 (+  5275us) log.cc:844] Serialized 52 byte log entry
0709 12:18:35.753045 (+   230us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:35.753096 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.753185 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.856918 (+103733us) raft_consensus.cc:1685] finished
0709 12:18:35.856941 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.857120 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24749,"prepare.queue_time_us":10478,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":53972,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":9603,"spinlock_wait_cycles":8758144,"thread_start_us":146754,"threads_started":1,"wal-append.queue_time_us":158445}
I20260709 12:18:35.889707 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.891877 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.887387 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.895507 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.896802 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.896813 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.901556 18366 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:35.904076 18373 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.904423 18373 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:35.906240 22655 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:35.906986 22655 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:35.907938 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:35.899785 23227 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:18:35.909211 19006 leader_election.cc:400] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13.
I20260709 12:18:35.909265 23227 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:35.909677 19006 leader_election.cc:403] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:35.909983 23227 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:35.910555 23227 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.910892 23168 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.900955 23145 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:35.911629 23168 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13.
I20260709 12:18:35.912081 23145 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:35.913379 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:35.915849 19037 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:35.916900 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:35.917501 19041 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:35.918737 22945 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:18:35.919426 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.885959 22819 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:35.908653 22831 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:35.905126 18373 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.924580 18295 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:35.924921 18373 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.830632 23221 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:35.926404 19037 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.922770 19041 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:35.902721 23232 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.894114 23194 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.799193 23182 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 12:18:35.901988 23231 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:35.981364 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.982093 18295 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.898963 23133 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:35.936595 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:35.935174 23221 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:35.938051 19041 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:35.906947 23197 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:18:36.009979 18297 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:35.888873 23236 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:18:35.976423 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:35.984803 18296 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.908214 22655 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:35.983770 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:35.992980 18338 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.557s (SENT) [suppressed 14 similar messages]
I20260709 12:18:35.992717 18373 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:36.004243 19042 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:18:36.009828 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:36.009495 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:36.014827 19037 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:18:35.986891 23161 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:18:36.036242 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.017241 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:36.040145 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:36.040606 19035 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:36.040722 19042 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.984274 23137 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:18:36.043376 23181 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.037168 22644 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:36.047593 22644 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:36.048087 22644 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.049455 19042 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:36.059805 18297 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:36.067708 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.068502 18447 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:36.069599 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.070640 18440 raft_consensus.cc:2410] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:18:36.073305 23196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59426 (request call id 1089) took 2377 ms. Trace:
I20260709 12:18:35.981840 23004 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:18:36.014170 23231 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:36.106145 18291 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [10.7-14.9]
W20260709 12:18:36.081785 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63236.1us
I20260709 12:18:36.106505 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.109932 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.073848 23196 rpcz_store.cc:276] 0709 12:18:33.702744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.703183 (+   439us) service_pool.cc:224] Handling call
0709 12:18:34.054514 (+351331us) ts_tablet_manager.cc:934] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
0709 12:18:35.200497 (+1145983us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:35.819642 (+619145us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:35.819809 (+   167us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:36.072393 (+252584us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:36.072725 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:36.073136 (+   411us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:36.073211 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":93367,"tablet-copy.queue_time_us":207616,"thread_start_us":183757,"threads_started":2}
W20260709 12:18:36.120918 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.086346 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:36.116575 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:36.042758 23161 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:36.134860 18300 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.135522 23161 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.146226 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:36.084300 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 74534.6us
I20260709 12:18:36.106063 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:36.118551 18296 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:36.138886 23010 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:36.080219 23137 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:36.149133 23137 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:36.149176 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:36.149900 23292 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 12:18:36.150079 23137 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.151417 23137 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.152215 23205 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.109598 23201 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:36.155148 18440 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:18:36.155532 18440 pending_rounds.cc:85] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 9
I20260709 12:18:36.155797 18440 pending_rounds.cc:107] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:18:36.156037 18440 raft_consensus.cc:2889] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.156584 23010 consensus_queue.cc:1243] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:36.157135 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.157421 18297 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:36.088815 18295 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:18:36.108522 23284 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.309s	user 0.001s	sys 0.006s
I20260709 12:18:36.079205 23080 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.164851 23080 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.169371 18443 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.149940 18446 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:36.174078 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.140676 18298 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:18:36.141494 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:36.177825 18298 pending_rounds.cc:85] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 5
I20260709 12:18:36.178192 18298 pending_rounds.cc:107] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:18:36.144600 18447 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 12:18:36.178469 18298 raft_consensus.cc:2889] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.073302 18449 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:36.158618 23135 raft_consensus.cc:2764] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:36.145974 23262 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.189957 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:36.189985 18444 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:18:36.190814 18293 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.190932 18444 pending_rounds.cc:85] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 11
I20260709 12:18:36.191540 18444 pending_rounds.cc:107] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:18:36.191861 18444 raft_consensus.cc:2889] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:36.193323 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.185600 23161 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:36.147585 23133 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.211766 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 315012us
I20260709 12:18:36.216652 18445 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.216495 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.207479 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.217482 18292 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:36.188026 22831 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.223763 18441 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.223843 22831 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.224915 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.225538 18441 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 8.
I20260709 12:18:36.226425 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.226495 22831 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:36.227628 23289 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:18:36.165766 23236 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:36.230412 18441 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:36.196827 23262 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.239722 18443 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:36.243597 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:36.237056 18446 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:36.192342 23196 tablet_copy_client.cc:806] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:18:36.243620 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:36.244721 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:36.248118 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:36.139633 22907 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Committing config change with OpId 10.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:36.251783 19037 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:36.190070 23197 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:36.253371 19036 raft_consensus.cc:2368] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
I20260709 12:18:36.234647 18298 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:18:36.255381 23221 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:18:36.210611 23201 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:36.280757 18446 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:36.258528 23149 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:36.228646 23168 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:18:36.283999 23078 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:36.311955 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 407966us
I20260709 12:18:36.257277 22831 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:18:36.247125 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.228768 23133 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.292622 23196 tablet_copy_client.cc:670] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:18:36.322252 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:36.312593 18293 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:36.325248 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:36.306897 23262 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:36.325112 23281 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.527s	user 0.000s	sys 0.006s
I20260709 12:18:36.228088 23231 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.310709 23149 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:36.238572 23236 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.312780 23140 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:36.281144 23197 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.326615 22819 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:36.360092 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:36.339576 23295 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:36.343801 23221 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:36.373373 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.382328 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.346861 22644 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.404747 22644 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.348964 23140 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:36.327457 23078 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.347700 23227 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.390167 23221 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:36.429945 23149 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.407650 23295 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.435602 23295 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.436107 23295 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.436565 23295 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.438596 23295 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.351816 23201 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:36.415360 23178 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:36.356516 23217 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:18:36.413609 23182 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.449821 23228 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:36.370604 22819 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.456662 23221 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.458455 23227 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.458941 18440 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.448837 23078 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:36.462813 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.416916 22655 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:36.499284 23293 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.459s	user 0.002s	sys 0.003s
I20260709 12:18:36.506220 23182 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:36.511656 23182 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.514161 23182 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:36.523607 18338 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.024s
W20260709 12:18:36.524076 18338 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.009s	sys 0.024s
I20260709 12:18:36.479636 23228 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:36.528821 18445 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:36.507279 22655 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:36.524461 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 531468us
W20260709 12:18:36.555380 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.552064 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 332038us
I20260709 12:18:36.542929 22655 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.502995 23140 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.575920 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:36.576225 22644 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.576591 19042 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:36.576718 22644 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.581043 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:36.588233 23111 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:36.588727 23111 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:36.589726 23111 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:36.600381 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.591037 23111 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:36.617667 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 442009us
I20260709 12:18:36.618490 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:36.623889 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:36.620148 23206 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:36.657632 18266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.020s	sys 0.028s
W20260709 12:18:36.658123 18266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.020s	sys 0.028s
W20260709 12:18:36.658535 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 521386us
W20260709 12:18:36.663576 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 331099us
W20260709 12:18:36.676662 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.678974 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:36.626452 23236 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.633888 23133 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.679513 22817 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:36.569962 23228 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.675992 23135 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:18:36.716851 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.699553 22694 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:36.680449 22945 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:36.683468 23182 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.815021 18296 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 6 to 9, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:36.810575 18293 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:18:36.811884 18298 pending_rounds.cc:85] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 5
I20260709 12:18:36.813378 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.675817 23262 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:18:36.833436 18298 pending_rounds.cc:107] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:36.833768 18298 raft_consensus.cc:2889] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.830152 18291 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:36.721215 22817 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.836988 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.842211 23182 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.804549 23129 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.847520 23129 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.850843 22819 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.851076 23182 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.845948 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.851413 22819 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.851851 22819 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:36.853751 22819 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:36.834719 22644 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:36.833163 22945 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:36.856109 23298 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.347s	user 0.003s	sys 0.005s
I20260709 12:18:36.856772 22945 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:36.826634 23100 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.863477 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.830862 23197 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:36.865269 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:36.843160 22817 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.842975 23228 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:36.888507 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.872040 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.889634 18294 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
W20260709 12:18:36.874730 23297 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
W20260709 12:18:36.875042 23299 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 12:18:36.875878 23301 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:18:36.876515 23140 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:36.841029 18441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.011s	sys 0.009s
I20260709 12:18:36.881217 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:36.896488 18441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.011s	sys 0.009s
I20260709 12:18:36.892035 18294 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:18:36.827732 23135 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.898365 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:36.899036 18291 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:36.899107 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:36.891804 19041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.008s	sys 0.019s
I20260709 12:18:36.859907 23236 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.900647 18297 raft_consensus.cc:2368] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
W20260709 12:18:36.899832 19041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.019s
W20260709 12:18:36.902279 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.902933 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.833171 22694 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:36.840471 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 509754us
I20260709 12:18:36.911955 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:36.891387 22947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.010s	sys 0.011s
I20260709 12:18:36.857942 23310 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 12:18:36.915247 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.927537 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:36.927564 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.908069 22916 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:36.872833 23100 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:36.900507 23197 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.941285 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.017272 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.052706 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.074383 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 144960us
I20260709 12:18:37.066772 18370 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Committing config change with OpId 10.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:37.087879 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1066) took 1173 ms. Trace:
I20260709 12:18:37.097378 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:37.098780 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:37.102120 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:37.064095 23318 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.926848 22947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.010s	sys 0.011s
I20260709 12:18:37.120163 18299 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:36.910953 23236 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.068392 22819 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:37.094336 23297 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:18:37.088541 19039 rpcz_store.cc:276] 0709 12:18:35.914508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.920112 (+  5604us) service_pool.cc:224] Handling call
0709 12:18:35.931667 (+ 11555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.934574 (+  2907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.934619 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.935230 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.420571 (+485341us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:36.420627 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.420718 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.087588 (+666870us) raft_consensus.cc:1685] finished
0709 12:18:37.087616 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.087805 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2431,"raft.queue_time_us":3647,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":25482,"thread_start_us":436082,"threads_started":1,"wal-append.queue_time_us":645766}
I20260709 12:18:37.144361 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2300) took 1522 ms. Trace:
I20260709 12:18:37.144917 18448 rpcz_store.cc:276] 0709 12:18:35.622241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.622370 (+   129us) service_pool.cc:224] Handling call
0709 12:18:35.623838 (+  1468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.624171 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.624195 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.937214 (+313019us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4c0030a500. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6711f644 00007f3c67146f2d 00007f3c67146b93 00007f3c68e7718b 00007f3c68e77cb5 00007f3c68e79004 00007f3c68e78729 00007f3c68e7dd96 00007f3c68e72adb
0709 12:18:35.938026 (+   812us) log.cc:844] Serialized 506 byte log entry
0709 12:18:36.286244 (+348218us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:36.286284 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.286349 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.144000 (+857651us) raft_consensus.cc:1685] finished
0709 12:18:37.144028 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.144286 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30042496,"thread_start_us":313594,"threads_started":1,"wal-append.queue_time_us":570975}
I20260709 12:18:37.147776 22819 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:37.149542 22819 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.150020 22819 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:37.153028 23308 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:18:37.153345 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.154009 19036 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:37.154973 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1119) took 1013 ms. Trace:
I20260709 12:18:37.155365 18442 rpcz_store.cc:276] 0709 12:18:36.141900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.142682 (+   782us) service_pool.cc:224] Handling call
0709 12:18:36.142990 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.143355 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.143384 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.143907 (+   523us) log.cc:844] Serialized 52 byte log entry
0709 12:18:36.562511 (+418604us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:36.562554 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.562637 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.154758 (+592121us) raft_consensus.cc:1685] finished
0709 12:18:37.154778 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.154903 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1654,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12315,"thread_start_us":343288,"threads_started":1,"wal-append.queue_time_us":622340}
I20260709 12:18:37.164324 19039 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:18:37.164743 22694 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.164923 19039 pending_rounds.cc:85] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 10
I20260709 12:18:37.165298 19039 pending_rounds.cc:107] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:18:37.165585 19039 raft_consensus.cc:2889] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:37.166553 22655 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.566s	user 0.004s	sys 0.003s
I20260709 12:18:37.094627 23182 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:37.100907 23197 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:37.187019 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.001s	sys 0.014s
I20260709 12:18:37.118844 23296 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 12:18:37.202258 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.001s	sys 0.014s
W20260709 12:18:37.208065 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 543242us
W20260709 12:18:37.210510 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.211109 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.194845 23312 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:18:37.172639 23181 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.166947 22655 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:37.186723 23194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.011s	sys 0.011s
I20260709 12:18:37.221146 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:37.230501 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 326722us
W20260709 12:18:37.186925 22907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.008s	sys 0.009s
I20260709 12:18:37.196045 23309 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.204164 23252 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.265643 23252 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.168118 23149 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:18:37.136879 23318 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.276196 22655 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.230903 18367 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:37.251075 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.285712 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.286041 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.252421 23100 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.286940 18375 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:37.257875 23135 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.259259 23332 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.264952 23236 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.223249 23181 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.294014 23181 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.211721 22694 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.295678 22694 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.298509 22694 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.290839 18367 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:37.292109 23344 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.292759 23332 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.310930 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.312057 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.317399 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.317982 18443 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:18:37.327888 18443 pending_rounds.cc:85] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:18:37.328258 18443 pending_rounds.cc:107] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:18:37.328565 18443 raft_consensus.cc:2889] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:37.330179 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.030s	sys 0.030s
W20260709 12:18:37.330662 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.030s	sys 0.030s
W20260709 12:18:37.241025 22907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.008s	sys 0.009s
W20260709 12:18:37.322314 18340 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.054s	sys 0.113s
W20260709 12:18:37.336758 18340 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.054s	sys 0.113s
W20260709 12:18:37.330986 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 670906us
W20260709 12:18:37.349761 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.300827 23145 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:37.376262 23111 consensus_meta.cc:287] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.001s
I20260709 12:18:37.351437 23194 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:37.413566 23331 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:18:37.432413 18298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.024s
W20260709 12:18:37.433017 18298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.024s
W20260709 12:18:37.422008 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 323266us
I20260709 12:18:37.435829 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:37.301241 23309 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.340866 22819 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:37.436602 18295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.005s
W20260709 12:18:37.455955 18295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.009s	sys 0.005s
I20260709 12:18:37.419283 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43588 (request call id 20) took 4934 ms. Trace:
I20260709 12:18:37.421454 23335 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.464928 18412 leader_election.cc:400] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
I20260709 12:18:37.468711 18412 leader_election.cc:403] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:37.422240 23334 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.470007 23334 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:37.472805 23334 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.422004 23291 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.414695 23145 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.335281 23344 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.503585 23231 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
W20260709 12:18:37.511294 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 165888us
I20260709 12:18:37.493971 23182 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.480316 23194 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:37.462419 23236 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:18:37.507778 22655 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.518394 23182 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:37.518849 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 272525us
I20260709 12:18:37.492839 23335 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.504660 23252 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.502164 23291 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.524034 23231 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.584199 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.584882 19037 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:18:37.543129 23182 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.588763 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:37.552757 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 476625us
I20260709 12:18:37.581040 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.537247 22655 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:37.511502 23338 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:18:37.590718 19039 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:37.593011 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.593881 19039 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:18:37.597745 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:37.536474 23232 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.533281 23197 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.555302 19041 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 6 to 9, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:37.607681 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.609282 22655 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.609699 22694 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.610370 22694 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.461993 22907 rpcz_store.cc:276] 0709 12:18:32.443953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.444367 (+   414us) service_pool.cc:224] Handling call
0709 12:18:33.371001 (+926634us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b18000d2a20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68eb773d 00007f3c68e82f3f 00007f3c68e826b3 00007f3c6de42d29 00007f3c673c11d4 00007f3c673c1162 00007f3c673c1091 00007f3c673c100c 00007f3c673c02c2 00007f3c671b748c
0709 12:18:34.199213 (+828212us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b58006998d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68dd80aa 00007f3c68e5a447 00007f3c68e7770a 00007f3c68e7876a 00007f3c68e7791b 00007f3c68e8303c 00007f3c68e826b3 00007f3c6de42d29 00007f3c673c11d4 00007f3c673c1162
0709 12:18:34.241254 (+ 42041us) log.cc:844] Serialized 506 byte log entry
0709 12:18:34.743654 (+502400us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b1800174e80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:35.029624 (+285970us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:35.129821 (+100197us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b1800174e80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68dd5b0b 00007f3c68e5ab63 00007f3c68e82752 00007f3c6de42d29
0709 12:18:37.378127 (+2248306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":463242,"raft.queue_time_us":856244,"raft.run_cpu_time_us":2408,"raft.run_wall_time_us":172549,"spinlock_wait_cycles":1593042560,"thread_start_us":47262,"threads_started":1,"wal-append.queue_time_us":192553,"wal-append.run_cpu_time_us":6008,"wal-append.run_wall_time_us":1666284}
I20260709 12:18:37.613166 22694 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.540500 23206 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:18:37.595826 23296 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.607933 23334 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.631157 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.606925 23338 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.634128 18412 leader_election.cc:400] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
I20260709 12:18:37.619853 23197 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.637902 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.599932 23231 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.646348 23231 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.643870 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.650318 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.653564 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.632189 23236 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.634337 23332 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:37.657068 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.654127 23231 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.636653 22694 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:37.617311 23232 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.660982 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.662544 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2310) took 1446 ms. Trace:
I20260709 12:18:37.679693 18440 rpcz_store.cc:276] 0709 12:18:36.215812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.215920 (+   108us) service_pool.cc:224] Handling call
0709 12:18:36.218874 (+  2954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.507706 (+288832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.662327 (+1154621us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400185bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:37.662484 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551652096}
W20260709 12:18:37.680696 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.672510 22916 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.002s
I20260709 12:18:37.661834 22819 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.686427 22916 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:37.686692 23353 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:37.686950 22819 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.687824 22819 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.688654 23145 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.689361 23335 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.689986 23335 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:37.690496 23335 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:37.691830 22819 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:37.686100 23350 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:18:37.638466 23334 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:37.662824 23339 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:18:37.623786 23194 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:37.695994 23325 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 12:18:37.742475 23228 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.849s	user 0.004s	sys 0.004s
I20260709 12:18:37.656894 22655 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:18:37.647923 23197 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.703676 22819 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:37.743201 22817 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.846s	user 0.005s	sys 0.003s
I20260709 12:18:37.643208 23338 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.709826 23309 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.696249 23336 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.702181 23359 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:37.727797 23344 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.699707 23365 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:37.696342 23317 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 12:18:37.727841 23340 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 12:18:37.730535 23221 consensus_meta.cc:287] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.223s	user 0.011s	sys 0.001s
I20260709 12:18:37.730228 23182 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.696569 23332 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.732808 23334 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.659919 23347 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.657330 23252 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.736016 23291 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.695472 23111 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:37.738667 23335 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.758952 23359 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.756620 22817 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.765265 22817 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.765436 23359 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.739156 23181 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.758652 23336 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.766700 18447 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:37.768069 23336 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.768541 23328 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.768772 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2585) took 1545 ms. Trace:
I20260709 12:18:37.769066 23328 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.769152 18441 rpcz_store.cc:276] 0709 12:18:36.223803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.231315 (+  7512us) service_pool.cc:224] Handling call
0709 12:18:36.231673 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.254371 (+ 22698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.254403 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.259209 (+  4806us) log.cc:844] Serialized 52 byte log entry
0709 12:18:36.896916 (+637707us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:36.896959 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.897027 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.768563 (+871536us) raft_consensus.cc:1685] finished
0709 12:18:37.768586 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.768717 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23917,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":20624,"thread_start_us":586819,"threads_started":1,"wal-append.queue_time_us":856501}
I20260709 12:18:37.770779 23332 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.772953 23332 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.773370 23197 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.773530 23347 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.775967 23347 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:37.740180 23348 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:18:37.660161 23236 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.778533 23129 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.779744 23236 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.634778 23296 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.747516 22819 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:37.781956 22819 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.748441 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.783705 18370 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:37.749406 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.753616 18375 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:37.743532 23232 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:37.715261 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 100118us
W20260709 12:18:37.759093 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 115522us
I20260709 12:18:37.787336 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.787791 19009 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:37.758843 23365 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.743721 23063 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:37.756140 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 95432.7us
I20260709 12:18:37.788833 23336 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.796429 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:37.800575 19039 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:37.805656 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.807405 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.796041 23357 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.809125 23357 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.809532 19037 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:37.811013 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.812067 19040 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:37.812335 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:37.813994 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:37.814471 23365 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.789592 23206 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.815402 23135 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:37.815873 23206 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.816565 23135 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:37.817901 23135 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:37.819806 23336 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.820502 22819 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:37.821026 23345 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 12:18:37.821679 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.822350 23336 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:37.822642 23341 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 12:18:37.818595 23206 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.823436 22907 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.823559 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.824002 22907 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.825017 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.825575 23338 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.826013 22907 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.786098 23252 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.826349 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2080) took 1015 ms. Trace:
I20260709 12:18:37.828086 23335 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:37.829489 23335 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:37.829919 23100 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.760215 23221 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.830773 23335 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:37.830618 23100 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.827486 23368 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.832080 23368 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:37.832487 18368 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:37.832721 23100 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.832834 23357 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.833151 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.832502 23368 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.833709 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.834501 19037 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:37.835281 23085 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.835877 18449 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:37.837263 23140 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.937s	user 0.009s	sys 0.002s
I20260709 12:18:37.835282 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.839815 19040 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:37.826776 18298 rpcz_store.cc:276] 0709 12:18:36.811161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.811308 (+   147us) service_pool.cc:224] Handling call
0709 12:18:36.811659 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.865750 (+ 54091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.865778 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.866328 (+   550us) log.cc:844] Serialized 52 byte log entry
0709 12:18:37.433565 (+567237us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.469132 (+ 35567us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.795863 (+326731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.795939 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.822149 (+ 26210us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:37.826076 (+  3927us) raft_consensus.cc:1685] finished
0709 12:18:37.826169 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.826295 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":381850,"prepare.queue_time_us":35636,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":355606,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":24199,"spinlock_wait_cycles":29690752,"thread_start_us":562620,"threads_started":1,"wal-append.queue_time_us":651748}
W20260709 12:18:37.826016 23333 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 12:18:37.801465 23063 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:37.841979 23063 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.842414 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1115) took 1267 ms. Trace:
I20260709 12:18:37.842782 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:37.843071 23063 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.843693 23312 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:37.841784 23368 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.844075 23312 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.844491 23312 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:37.845677 23368 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:37.847051 23316 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.709s	user 0.000s	sys 0.006s
I20260709 12:18:37.842823 19038 rpcz_store.cc:276] 0709 12:18:36.575496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.582135 (+  6639us) service_pool.cc:224] Handling call
0709 12:18:36.582520 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.582925 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.582948 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.583497 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.917126 (+333629us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:18:37.001190 (+ 84064us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.001529 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.001635 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.825732 (+824097us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b2400307838. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dce09d 00007f3c68a0010e 00007f3c68a1788c 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221
0709 12:18:37.837667 (+ 11935us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b18001aa700. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f557 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:37.842243 (+  4576us) raft_consensus.cc:1685] finished
0709 12:18:37.842263 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.842364 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":105,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":8009,"raft.queue_time_us":63302,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":22473,"spinlock_wait_cycles":59705088,"thread_start_us":333319,"threads_started":1,"wal-append.queue_time_us":767604}
I20260709 12:18:37.849536 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 838) took 1603 ms. Trace:
I20260709 12:18:37.849475 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:37.850878 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.829993 23344 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.826092 23338 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.828334 23252 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.855535 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.862775 18373 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 8.
I20260709 12:18:37.856298 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:37.864475 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:37.854995 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:37.860234 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.849969 19035 rpcz_store.cc:276] 0709 12:18:36.246202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.247097 (+   895us) service_pool.cc:224] Handling call
0709 12:18:36.284361 (+ 37264us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.284809 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.284850 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.291837 (+  6987us) log.cc:844] Serialized 52 byte log entry
0709 12:18:36.701809 (+409972us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:36.825688 (+123879us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.835932 (+ 10244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.836027 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.725337 (+889310us) spinlock_profiling.cc:243] Waited 7.93 ms on lock 0x7b180049d080. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:37.846404 (+121067us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:37.849301 (+  2897us) raft_consensus.cc:1685] finished
0709 12:18:37.849330 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.849481 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43022,"prepare.queue_time_us":62485,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":43414,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":19749,"spinlock_wait_cycles":24837760,"thread_start_us":393159,"threads_started":1,"wal-append.queue_time_us":608531}
I20260709 12:18:37.861164 23336 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.869431 18442 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
W20260709 12:18:37.870713 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66443.2us
I20260709 12:18:37.862617 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.876642 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.877476 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:37.832221 23221 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:37.878460 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.879865 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.880512 18443 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:37.857714 23111 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.882936 19040 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:37.875120 19037 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:37.885582 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.886291 18370 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 9: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:37.886426 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.887059 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.886432 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:37.889202 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.889460 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.877987 23372 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:18:37.889948 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.919469 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.870149 23338 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.920109 23181 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.921056 23084 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.921453 23084 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.921630 23338 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:37.921823 23084 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.869450 23140 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:37.913606 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 85f32970b7374dcf9b9bac47791c71e3: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:18:37.883852 23344 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 12:18:37.914309 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52057.4us
I20260709 12:18:37.887542 23365 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:37.916575 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:37.910485 23111 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.913069 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.918581 18373 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:18:37.933674 19042 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:37.923466 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.933966 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.918558 23312 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.924147 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.941149 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:37.941721 18444 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:37.942977 19035 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:18:37.924990 18367 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:18:37.913061 23182 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:37.868597 23135 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:37.878168 23335 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:37.878579 23313 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.932389 23181 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:37.912803 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:37.938865 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.951097 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.958706 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:37.887621 23063 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:37.960489 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:37.921913 23197 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:18:37.966631 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.972885 18370 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:37.983462 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.980713 18375 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 8.
I20260709 12:18:37.984540 19037 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:18:37.967422 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:37.989295 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.986176 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.991660 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:37.985239 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.879189 23336 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:37.973202 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:37.993433 19042 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:37.993646 18375 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:18:37.979821 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.985208 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:37.933468 23194 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:18:37.981271 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
W20260709 12:18:37.973337 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.982532 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.992641 18370 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.967718 23135 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:38.007890 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:37.999166 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:37.973579 23353 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.000093 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:37.937271 23365 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.012404 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.012697 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.013195 18291 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.014284 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.015373 18298 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:37.969841 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2072) took 2081 ms. Trace:
I20260709 12:18:37.887051 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.005141 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.005362 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:37.971185 23313 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.021235 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.028256 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:38.031203 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 56397.9us
I20260709 12:18:38.004698 22817 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.996315 22916 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:38.031101 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.032060 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:38.018685 18374 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:38.046830 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:37.890525 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:38.040743 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.049270 18293 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:38.041996 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.043915 18295 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:18:38.050382 18295 pending_rounds.cc:85] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 10
I20260709 12:18:38.050623 18295 pending_rounds.cc:107] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:18:38.050830 18295 raft_consensus.cc:2889] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:38.006832 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.051961 18367 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:38.054616 18370 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:38.017961 18371 rpcz_store.cc:276] 0709 12:18:35.888181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.888342 (+   161us) service_pool.cc:224] Handling call
0709 12:18:35.903253 (+ 14911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.903448 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:35.903700 (+   252us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.674894 (+771194us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.849096 (+1174202us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b180029b180. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:37.969431 (+120335us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.969496 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:37.969564 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.969649 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.969778 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3598,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":474748,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":30885,"spinlock_wait_cycles":10007296,"thread_start_us":890716,"threads_started":3,"wal-append.queue_time_us":665990}
I20260709 12:18:38.010143 23135 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:37.986023 23063 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:37.982613 23181 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:38.059866 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 169455us
I20260709 12:18:38.071908 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:38.028141 23313 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.099957 23004 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:38.106930 23004 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:38.110513 23136 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.110984 23136 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:38.111653 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 79674.4us
I20260709 12:18:38.118125 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.045387 22817 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.059408 22916 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.123605 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:38.125079 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:38.097447 23308 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:18:38.065316 23309 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.086752 23100 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:38.062409 18371 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:38.114581 23168 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.125985 22817 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.125269 22916 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.129354 23309 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.120033 23307 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:18:38.161033 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 61627.4us
I20260709 12:18:38.159663 23395 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.136265 23194 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:38.138491 23360 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.171437 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:38.166567 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.190966 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.172262 18374 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.195994 18291 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:38.206454 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.206892 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.207554 18294 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:38.208313 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.203191 19041 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:38.206261 18291 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:38.210455 18291 pending_rounds.cc:85] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 6
I20260709 12:18:38.210877 18291 pending_rounds.cc:107] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:38.211292 18291 raft_consensus.cc:2889] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:38.162269 23309 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.167682 23262 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.228235 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.229758 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.220448 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.171017 23360 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.241045 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.241822 18373 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:38.242496 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:38.175594 23374 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:18:38.182644 23373 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 12:18:38.191742 23370 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:18:38.236795 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.180966 23181 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.216861 23307 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:38.175902 23303 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.370s	user 0.007s	sys 0.021s
I20260709 12:18:38.238785 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.243116 18374 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:38.259713 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.245414 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.247000 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.173676 23111 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.179258 23312 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.249881 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.182542 23336 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.188138 23168 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:38.228859 22644 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.263602 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:38.263950 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.279557 18371 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.267438 18294 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:38.238875 23365 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.248956 23360 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.285451 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.286974 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.287680 18294 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:38.170848 23395 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.261270 23063 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.298699 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.300203 18300 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:38.301302 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.302672 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.278681 23168 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.246727 23262 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.310611 18294 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:38.289283 23312 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.312714 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.313614 18294 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:18:38.314548 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86550.6us
I20260709 12:18:38.283192 22644 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.266651 23307 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.323509 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.324301 18294 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:38.324671 18371 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:38.325580 23405 ts_tablet_manager.cc:933] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
I20260709 12:18:38.330073 23262 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.340219 18300 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
W20260709 12:18:38.344861 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 137303us
I20260709 12:18:38.346079 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:38.315388 23312 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.355645 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:38.360661 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.365166 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:38.369571 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:38.358707 23217 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:38.375136 23393 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:38.361385 23380 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:38.361546 23365 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:38.373240 23100 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:38.400902 23161 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.403016 23405 tablet_copy_client.cc:323] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.65:37771
I20260709 12:18:38.371147 23335 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:18:38.392364 23396 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:38.410729 23353 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.401011 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:38.418586 22817 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.423770 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2329) took 1186 ms. Trace:
I20260709 12:18:38.389472 23135 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.390621 22916 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.430919 18368 rpcz_store.cc:276] 0709 12:18:37.237348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.237476 (+   128us) service_pool.cc:224] Handling call
0709 12:18:37.237768 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.832875 (+595107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.423458 (+590583us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400116054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:38.423689 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367283584}
I20260709 12:18:38.421401 23380 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.420446 23217 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:38.432711 18373 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:38.427536 23161 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.437598 23161 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.441134 22817 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.441737 22644 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.441630 23313 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.436301 23380 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:38.443485 22916 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.443624 23313 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.444401 23307 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.428613 23100 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.445305 23135 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.432458 23395 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.446542 23100 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:38.448458 23084 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.003s
I20260709 12:18:38.448784 18298 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:38.449050 23084 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.451805 18310 tablet_copy_service.cc:140] P e23c0ce8d89940da8eac6a6002e349c5: Received BeginTabletCopySession request for tablet 8903d70080d24236be955433a92791c7 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:43604)
I20260709 12:18:38.452037 18294 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:38.452322 18310 tablet_copy_service.cc:161] P e23c0ce8d89940da8eac6a6002e349c5: Beginning new tablet copy session on tablet 8903d70080d24236be955433a92791c7 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:43604: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-8903d70080d24236be955433a92791c7
I20260709 12:18:38.459771 23307 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:38.463070 23307 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.464382 23084 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.465888 23307 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.437953 23217 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.454172 23100 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:38.473881 23307 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:38.474354 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:38.479394 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:38.481941 18310 tablet_copy_source_session.cc:215] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:18:38.483714 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.484745 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.485376 19039 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:38.487125 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.487365 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:38.487910 19036 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:38.488660 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:38.488924 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.489303 19039 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.475108 23410 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:18:38.491415 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.492558 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:38.493124 18370 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.493594 23360 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.495658 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:38.497177 23309 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.493124 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:38.498353 18374 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.467634 23168 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.500914 19041 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:38.502198 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 101322us
I20260709 12:18:38.502976 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:38.476990 23395 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.504601 19039 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:38.506201 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:38.506769 23309 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.530344 23410 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:38.532725 23410 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.533141 23413 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.533717 23413 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.534870 23410 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.507659 18374 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:38.507608 18370 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:38.481376 23400 raft_consensus.cc:2764] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:18:38.510053 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.515877 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:38.507169 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.481426 22916 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.538484 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:38.478255 23100 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.542004 23360 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.541059 19041 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:18:38.544090 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.549305 23318 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.551410 22694 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.551972 22694 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:38.554049 22694 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.506781 23335 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.556740 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:38.561122 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:38.565930 22644 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:38.566672 22644 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.567559 23161 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.564779 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:38.558827 23400 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:38.572177 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:38.557897 23414 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:38.564422 23318 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:18:38.572886 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.568761 22694 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.582526 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:38.569602 23371 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.595432 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:38.571128 23413 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.540293 23395 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.572798 23063 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:38.569384 23380 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.574817 23400 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.582065 23347 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:38.572743 23335 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.581255 23318 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.597066 23161 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:38.612738 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 96947.4us
I20260709 12:18:38.621950 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:38.623044 18372 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:38.606894 22694 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:38.627245 22644 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.611356 23227 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:38.644078 23168 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.619441 23318 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:38.641660 23217 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.616660 23400 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.624001 23335 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.650384 23395 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.650990 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.663312 23413 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.662572 23380 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.655488 23227 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:38.665269 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:38.667482 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72134.1us
I20260709 12:18:38.678748 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.678876 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:38.681793 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.682487 18299 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:38.683667 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.685021 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.687521 23307 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:38.681428 23309 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:38.677546 23413 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:38.713438 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.688275 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.715744 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:38.687604 23010 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:38.722577 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.699867 23328 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.728060 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.729837 23347 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:38.749711 23347 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:38.749984 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.697077 23335 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.716810 23395 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.729635 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.750690 23347 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.751128 23395 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.751772 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.735263 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.735576 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.754004 18369 raft_consensus.cc:2368] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:38.742656 23217 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.745007 23296 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.695636 22694 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:38.748284 23227 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.789494 22694 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.789870 22694 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:38.722054 23309 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:38.738612 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:38.718864 23307 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.725901 23413 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.709370 23400 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.779767 23417 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:38.781448 22916 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.785415 23136 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.789944 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:38.822037 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 83520.1us
I20260709 12:18:38.828696 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:38.829789 18297 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.786665 23318 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.831729 18415 leader_election.cc:400] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:38.836382 18415 leader_election.cc:403] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:38.839340 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.840075 18366 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:38.840270 18369 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:38.835362 18299 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:18:38.842751 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 216555us
I20260709 12:18:38.842689 18295 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:38.844131 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:38.826920 23296 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:38.821062 23417 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:38.842047 23231 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:18:38.831283 23232 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:38.841832 23422 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.847282 23421 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.847518 23371 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.857008 23136 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.857445 23422 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.879263 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.890699 23395 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.896347 23318 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.857698 23232 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.897902 23182 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:38.898675 23232 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.892726 23307 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:38.886490 23422 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.894207 18292 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:38.903450 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.903568 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.863023 23371 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:38.909261 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:38.910115 18373 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:38.911824 18292 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:18:38.915620 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:38.916504 18294 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:38.865403 23421 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:38.934252 18292 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:18:38.936098 19069 heartbeater.cc:646] Failed to heartbeat to 127.16.193.126:46027 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.16.193.126:46027 timed out after 15.000s (SENT)
I20260709 12:18:39.011932 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:39.012583 23396 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:39.013056 23396 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:39.014748 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:39.014911 23351 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:39.015251 23365 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:39.015626 23365 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:39.015566 23351 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:39.016011 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:39.016539 23365 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.017534 23365 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:39.017740 23351 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:39.014767 19041 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:38.907004 23371 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.024111 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:39.024636 19043 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.859889 23309 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.878643 23417 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:38.961457 23312 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:38.954010 23100 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:38.979086 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:39.045686 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66728.8us
I20260709 12:18:38.999997 18375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39264
I20260709 12:18:39.047830 18375 raft_consensus.cc:606] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Received request to transfer leadership
I20260709 12:18:39.048240 18375 raft_consensus.cc:614] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:38.966754 23400 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.001897 22907 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:18:39.007054 23421 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:39.027783 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.028625 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.029261 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.078415 18444 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:39.078712 18441 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:39.079624 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.079833 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.080166 18441 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:39.078459 18445 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:39.081204 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.082309 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:39.082505 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.044178 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.083012 18445 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.045771 19043 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:39.032276 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.084729 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:39.084991 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.085996 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.086862 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:39.083520 18440 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:39.080722 18444 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:38.989694 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.083074 18441 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:39.092089 18441 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:39.093503 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:39.094535 19043 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:39.103819 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 854) took 1221 ms. Trace:
I20260709 12:18:39.037972 23181 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:39.110394 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 109093us
I20260709 12:18:39.104388 19044 rpcz_store.cc:276] 0709 12:18:37.882321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.882467 (+   146us) service_pool.cc:224] Handling call
0709 12:18:37.882853 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.159470 (+276617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.159505 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.160009 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:18:38.347623 (+187614us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:38.347674 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.065644 (+717970us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540071e8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:39.065710 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.065775 (+    65us) raft_consensus.cc:1685] finished
0709 12:18:39.065796 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.103757 (+ 37961us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19553,"raft.queue_time_us":49688,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":20855,"spinlock_wait_cycles":294640000,"thread_start_us":287935,"threads_started":2,"wal-append.queue_time_us":269794}
I20260709 12:18:39.041198 23338 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.055900 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.060523 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.060781 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.064980 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:39.039552 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 192775us
I20260709 12:18:39.057014 23194 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:39.053285 23296 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:39.054742 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:38.858767 22694 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.154004 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.150660 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.225600 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.263943 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 135788us
W20260709 12:18:39.236063 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57904.1us
I20260709 12:18:39.265897 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.233491 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:39.267200 18298 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.267326 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:39.268047 18374 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:39.268553 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.133715 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.240478 18297 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:39.270393 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:39.140650 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:39.272080 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.232874 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:39.266063 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:39.239637 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 154980us
I20260709 12:18:39.272248 18296 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:39.274685 19041 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:18:39.265102 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.275785 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:39.275960 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.277848 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:39.279923 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:39.280586 19043 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:39.281886 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.282521 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.243835 18445 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:39.283191 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.285830 19044 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:39.263649 18375 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 6 to 9, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:39.287894 18374 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:39.289644 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.290226 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:39.293895 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:39.294391 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.295796 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.296593 18366 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:39.298381 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:39.300100 18296 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:39.302194 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:39.303090 18446 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:39.188372 23196 tablet_copy_client.cc:538] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:18:39.304991 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:39.282025 18294 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:39.367348 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:39.368878 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:39.373323 18370 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:39.177030 23419 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.261051 23194 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:39.377831 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 83506.3us
I20260709 12:18:39.381261 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.187292 23421 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:39.277618 22907 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:39.384584 19035 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:39.387549 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:39.302520 23227 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.375129 23145 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:39.394079 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:39.139865 23181 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.385404 23419 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.379734 23351 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.366751 22945 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:39.384850 23424 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:18:39.169638 23423 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.247s
I20260709 12:18:39.385994 23421 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:39.394649 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.380362 23312 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.423401 23405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8903d70080d24236be955433a92791c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:39.384284 23194 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:39.395442 22694 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.383399 23085 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:39.395677 23347 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:39.395898 22907 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:39.419378 23421 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:39.414355 23425 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:18:39.419955 23419 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.414896 22945 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.409679 23197 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.453361 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1080) took 1523 ms. Trace:
I20260709 12:18:39.406129 23227 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.417158 23063 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.425092 22819 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.452550 22945 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:39.478087 23231 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.003s
I20260709 12:18:39.439946 23085 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.416427 23296 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.443090 23347 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.498623 23231 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:39.504010 23201 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:39.482466 23312 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:39.511502 23441 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:18:39.536594 23426 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:18:39.586035 23434 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:18:39.585023 23437 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:18:39.590286 23422 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.682s	user 0.006s	sys 0.005s
W20260709 12:18:39.628926 18246 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3da7ad8fa20a4591963b660e4c848e7f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } } pending_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:39.630271 18246 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:18:39.630826 18246 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:18:39.637666 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:39.639582 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:39.641346 18297 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:39.641872 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1054) took 1688 ms. Trace:
I20260709 12:18:39.642351 18367 rpcz_store.cc:276] 0709 12:18:37.952791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.062929 (+110138us) service_pool.cc:224] Handling call
0709 12:18:38.118273 (+ 55344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:38.118703 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.118744 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:38.119304 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:18:38.750915 (+631611us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18002af700. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:39.040950 (+290035us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:39.041003 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.041102 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.640946 (+599844us) raft_consensus.cc:1685] finished
0709 12:18:39.641004 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.641588 (+   584us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9617,"spinlock_wait_cycles":4343168,"thread_start_us":289824,"threads_started":1,"wal-append.queue_time_us":1055172}
I20260709 12:18:39.650789 18297 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:39.651711 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 910) took 1407 ms. Trace:
W20260709 12:18:39.655567 18442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.019s	sys 0.005s
I20260709 12:18:39.421268 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:39.656688 18448 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
W20260709 12:18:39.658774 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 254781us
I20260709 12:18:39.453907 19037 rpcz_store.cc:276] 0709 12:18:37.930252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.931435 (+  1183us) service_pool.cc:224] Handling call
0709 12:18:37.932177 (+   742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.987479 (+ 55302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.453035 (+1465556us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b5400747f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:39.453290 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95081728}
I20260709 12:18:39.674568 22916 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.501039 22907 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.507877 23145 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:39.693794 23337 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:39.694664 23318 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:39.697726 18297 rpcz_store.cc:276] 0709 12:18:38.244002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.642471 (+1398469us) service_pool.cc:224] Handling call
0709 12:18:39.643092 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.651352 (+  8260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.651616 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:39.701285 18442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.019s	sys 0.006s
I20260709 12:18:39.690979 23085 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:39.707798 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 74702.7us
I20260709 12:18:39.695040 23100 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:39.695761 23344 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:18:39.721812 22945 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.723135 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:39.725018 23227 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.727862 18297 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:39.729236 23318 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.731595 23337 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.732949 23347 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.599695 23422 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.735471 23441 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:39.735905 23441 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:39.736124 23422 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.736317 23441 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.740440 23414 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:39.741190 23420 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:39.741701 23420 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:39.724593 22916 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.749313 23423 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.750017 23423 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.718516 23100 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:39.751322 23420 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.752996 23181 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.752801 23100 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.753113 22916 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:39.753495 23423 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:39.727916 23145 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.754688 23100 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.734615 23337 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:39.756552 23318 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:39.728901 23312 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.761649 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.762375 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 920) took 1363 ms. Trace:
I20260709 12:18:39.762823 18299 rpcz_store.cc:276] 0709 12:18:38.399170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.722266 (+323096us) service_pool.cc:224] Handling call
0709 12:18:39.761876 (+1039610us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b54002751d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:39.762133 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183384448}
I20260709 12:18:39.728930 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 916) took 1460 ms. Trace:
I20260709 12:18:39.765070 18297 rpcz_store.cc:276] 0709 12:18:38.268762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.726805 (+1458043us) service_pool.cc:224] Handling call
0709 12:18:39.727460 (+   655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.728539 (+  1079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.728841 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:39.753209 23420 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.769582 18299 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:39.769214 22945 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:39.776147 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 394815us
I20260709 12:18:39.777609 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.774231 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 922) took 1354 ms. Trace:
I20260709 12:18:39.779959 18299 rpcz_store.cc:276] 0709 12:18:38.416545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.764532 (+1347987us) service_pool.cc:224] Handling call
0709 12:18:39.769092 (+  4560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.770221 (+  1129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.771165 (+   944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:39.787938 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 911) took 1543 ms. Trace:
I20260709 12:18:39.788967 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 926) took 1157 ms. Trace:
I20260709 12:18:39.789479 18297 rpcz_store.cc:276] 0709 12:18:38.631517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.781528 (+1150011us) service_pool.cc:224] Handling call
0709 12:18:39.788103 (+  6575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.788534 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.788569 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:39.788642 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:39.788691 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.788761 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.788880 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:39.709439 23201 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:39.774032 23318 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:39.798058 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.800879 23231 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:39.802198 23201 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.803393 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 382048us
I20260709 12:18:39.803498 22945 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:39.810856 22945 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.778939 23438 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:39.805436 23365 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.787s	user 0.014s	sys 0.002s
I20260709 12:18:39.812482 22945 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:39.816895 23201 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:39.825474 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.827371 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.811362 23231 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.831678 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54326.7us
I20260709 12:18:39.830173 18297 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:39.833052 23318 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.835168 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.846630 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.847307 23438 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:39.848567 23447 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.849308 23447 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.850281 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:39.851178 23085 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.851383 23168 raft_consensus.cc:2764] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:18:39.853367 23447 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.848207 23438 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.853793 23136 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.977s	user 0.004s	sys 0.012s
I20260709 12:18:39.854617 23427 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:39.855430 23438 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:39.857187 23371 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.834s	user 0.004s	sys 0.011s
I20260709 12:18:39.858925 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.857931 23371 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:39.860086 23413 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:39.863835 23417 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.886770 18371 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:39.871608 23421 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.900940 23413 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:39.788511 18294 rpcz_store.cc:276] 0709 12:18:38.244288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.648142 (+1403854us) service_pool.cc:224] Handling call
0709 12:18:39.648655 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.649173 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:39.649715 (+   542us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.787482 (+137767us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:39.787545 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:39.787601 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.787697 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.787841 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59124,"prepare.queue_time_us":48851,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":3329,"raft.queue_time_us":68930,"thread_start_us":68557,"threads_started":1,"wal-append.queue_time_us":137552}
I20260709 12:18:39.876992 18293 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:18:39.881224 23396 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.868s	user 0.012s	sys 0.006s
W20260709 12:18:39.882246 23430 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.407s	user 0.004s	sys 0.000s
I20260709 12:18:39.884261 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2503) took 1142 ms. Trace:
I20260709 12:18:39.905447 18295 rpcz_store.cc:276] 0709 12:18:38.741994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.749713 (+  7719us) service_pool.cc:224] Handling call
0709 12:18:39.883909 (+1134196us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b5400002154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:39.884085 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211454336}
I20260709 12:18:39.907137 18291 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:39.887369 23427 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.889561 23231 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.867738 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2499) took 1107 ms. Trace:
I20260709 12:18:39.893775 23217 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:39.929442 18440 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:39.846658 23318 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:39.857825 18298 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.011s
I20260709 12:18:39.904678 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2500) took 1166 ms. Trace:
I20260709 12:18:39.909014 23396 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.916244 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 128827us
W20260709 12:18:39.919062 23433 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.471s	user 0.000s	sys 0.014s
I20260709 12:18:39.918366 18194 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:39.921712 18297 rpcz_store.cc:276] 0709 12:18:38.737218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.790764 (+1053546us) service_pool.cc:224] Handling call
0709 12:18:39.791201 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.842846 (+ 51645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.844432 (+  1586us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:39.922804 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102125us
I20260709 12:18:39.923344 23182 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:39.924461 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65709.8us
I20260709 12:18:39.842623 23441 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:39.932243 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 161306us
I20260709 12:18:39.932220 18293 rpcz_store.cc:276] 0709 12:18:38.738220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.876043 (+1137823us) service_pool.cc:224] Handling call
0709 12:18:39.876487 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.904407 (+ 27920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.904626 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:39.939828 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:39.941793 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 183887us
I20260709 12:18:39.920491 23413 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:39.929383 23443 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:39.950547 19043 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:39.952592 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.956001 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 909) took 1712 ms. Trace:
I20260709 12:18:39.957027 23427 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:39.958465 18442 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:39.960776 18446 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:39.964533 18443 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:39.956914 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:39.938934 23217 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:39.967846 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:39.968940 23194 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.007s
I20260709 12:18:39.969300 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:39.969408 23194 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.970534 23262 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.970902 19040 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:39.971293 23262 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.973321 23371 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:39.976451 19043 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:39.976091 23443 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:39.976506 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:39.978942 19040 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:39.980420 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.981638 23428 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.983552 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:39.984405 23428 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:39.956509 18300 rpcz_store.cc:276] 0709 12:18:38.243651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.017415 (+773764us) service_pool.cc:224] Handling call
0709 12:18:39.017726 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.018119 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:39.018538 (+   419us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.366723 (+348185us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.640287 (+273564us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.640615 (+   328us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:39.640731 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.640919 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.918220 (+277301us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:39.955435 (+ 37215us) raft_consensus.cc:1685] finished
0709 12:18:39.955572 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.955777 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34312,"prepare.queue_time_us":784,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":101914,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":4542,"spinlock_wait_cycles":73381760,"thread_start_us":250030,"threads_started":1,"wal-append.queue_time_us":393551}
W20260709 12:18:40.001884 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.018172 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.019124 18440 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.026688 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:40.027336 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:40.028026 19040 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:40.027892 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.028920 18366 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:40.030525 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:40.033890 18370 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:18:40.034950 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.036466 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.037140 18372 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.037628 18440 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
I20260709 12:18:40.039784 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.040673 23428 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:40.041077 23428 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.041447 23428 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:40.044695 18198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 295) took 13746 ms (client timeout 15000 ms). Trace:
I20260709 12:18:40.052245 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:40.053211 23309 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:18:40.054021 23136 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:40.054287 23309 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.054430 23136 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.054751 23309 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:40.054813 23136 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:40.056243 23217 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:40.070231 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 86817.2us
I20260709 12:18:40.007704 23420 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:40.097673 22945 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.097272 18194 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 13 to 14, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:40.045253 18198 rpcz_store.cc:269] 0709 12:18:26.297653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.297814 (+   161us) service_pool.cc:224] Handling call
0709 12:18:40.044361 (+13746547us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:33.717921 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:33.840531 (+122610us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:33.840588 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:18:33.840608 (+    20us) tablet.cc:662] Decoding operations
0709 12:18:33.842791 (+  2183us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:33.842896 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:33.842944 (+    48us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:18:34.044356 (+201412us) tablet.cc:701] Row locks acquired
0709 12:18:34.044403 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:18:34.044590 (+   187us) write_op.cc:270] Start()
0709 12:18:34.044742 (+   152us) write_op.cc:276] Timestamp: P: 1783599514044515 usec, L: 0
0709 12:18:34.044776 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:18:34.096405 (+ 51629us) log.cc:844] Serialized 11008 byte log entry
0709 12:18:34.714123 (+617718us) op_driver.cc:464] REPLICATION: finished
0709 12:18:34.808056 (+ 93933us) write_op.cc:301] APPLY: starting
0709 12:18:34.811839 (+  3783us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:35.035125 (+223286us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:35.042882 (+  7757us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:35.257776 (+214894us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:35.258180 (+   404us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 12:18:35.258225 (+    45us) write_op.cc:312] APPLY: finished
0709 12:18:35.258728 (+   503us) log.cc:844] Serialized 244 byte log entry
0709 12:18:35.258909 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:35.259128 (+   219us) write_op.cc:454] Released schema lock
0709 12:18:35.259193 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:35.291391 (+ 32198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:35.291422 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":267682,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":53753,"apply.run_cpu_time_us":39330,"apply.run_wall_time_us":490642,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121055,"num_ops":28,"prepare.queue_time_us":254914,"prepare.run_cpu_time_us":34855,"prepare.run_wall_time_us":794300,"raft.queue_time_us":91688,"raft.run_cpu_time_us":6225,"raft.run_wall_time_us":11220,"replication_time_us":669183,"thread_start_us":479001,"threads_started":4,"wal-append.queue_time_us":434840,"wal-append.run_cpu_time_us":20017,"wal-append.run_wall_time_us":1715113}]]}
I20260709 12:18:40.078414 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.112061 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.113500 18440 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.131408 19043 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:18:40.112058 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.134577 18440 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:40.137668 23420 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.138294 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.139217 18448 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:40.140074 23420 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.141613 23443 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.141208 23309 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.145768 23196 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:18:40.149194 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2495) took 1477 ms. Trace:
I20260709 12:18:40.150058 23441 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.149569 18299 rpcz_store.cc:276] 0709 12:18:38.672326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.780894 (+1108568us) service_pool.cc:224] Handling call
0709 12:18:39.813175 (+ 32281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.835660 (+ 22485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.835699 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.857162 (+ 21463us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.000760 (+143598us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.000796 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.000862 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.148946 (+148084us) raft_consensus.cc:1685] finished
0709 12:18:40.148967 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.149111 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16880,"raft.queue_time_us":7704,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":149,"thread_start_us":143177,"threads_started":1,"wal-append.queue_time_us":225550}
I20260709 12:18:40.140383 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.153028 18449 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:40.156271 18371 raft_consensus.cc:2368] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:40.160872 23396 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.165364 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:40.169978 23194 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.174563 23414 raft_consensus.cc:2764] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:40.175638 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.171304 23217 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.177462 23217 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.178768 22945 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.172205 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:40.179919 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.181881 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.182931 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.184482 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.189256 18447 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:40.190771 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.192838 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:40.202564 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 200488us
I20260709 12:18:40.197911 18446 raft_consensus.cc:2376] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10. Re-sending same reply.
I20260709 12:18:40.211443 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.212105 18443 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:40.212937 18441 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:40.218240 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.217512 18372 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:40.194604 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 10 to 12, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:40.220674 18442 raft_consensus.cc:2410] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:40.222397 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.227596 18440 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:40.228508 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.229557 18370 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:40.229892 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.205437 23309 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:40.319553 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 95363.2us
W20260709 12:18:40.342438 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.349790 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:40.363935 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.373785 18445 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.230571 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:40.245819 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.358956 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 92352.9us
W20260709 12:18:40.249024 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.325623 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.333727 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:40.266081 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 94007.5us
I20260709 12:18:40.301352 18441 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.331637 18194 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 10, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:40.379522 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.376003 18442 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:40.348191 22945 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.382367 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.384160 23227 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:40.386128 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:40.374455 23197 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.062s	sys 0.091s
I20260709 12:18:40.389616 18440 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:40.268963 23421 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.302487 23217 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.266206 23371 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.388268 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.407663 18449 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:40.349932 23458 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.394207 23262 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:40.395027 22945 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.399039 18447 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:18:40.404503 23197 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.063s	sys 0.091s
I20260709 12:18:40.265446 23136 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.406541 23421 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.427363 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.430711 19043 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:40.431458 22945 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.414878 23197 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:40.446133 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 112098us
I20260709 12:18:40.425734 18194 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 4 to 9, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 9 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:40.447482 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.450791 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.449612 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.448375 23417 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:18:40.452920 18373 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:40.455173 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:40.456574 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.437459 23458 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.434410 23421 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.460132 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2102) took 1109 ms. Trace:
W20260709 12:18:40.464399 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 78533.9us
I20260709 12:18:40.435925 23428 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:40.439000 23371 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.460633 18375 rpcz_store.cc:276] 0709 12:18:39.262735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.262882 (+   147us) service_pool.cc:224] Handling call
0709 12:18:39.263271 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.263514 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:39.764215 (+500701us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.243530 (+479315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:40.310188 (+ 66658us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:40.331077 (+ 20889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.339995 (+  8918us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.371692 (+ 31697us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136738,"raft.queue_time_us":149454,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":45596,"thread_start_us":306397,"threads_started":1,"wal-append.queue_time_us":395065}
I20260709 12:18:40.464679 22945 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.475577 22945 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.475951 22945 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.477741 18449 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.481808 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.487429 23458 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.493971 18447 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:40.496457 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:40.497643 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57491.4us
I20260709 12:18:40.500497 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.505537 18440 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:40.504419 18375 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:40.509223 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:40.510155 23396 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:40.510560 23396 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.510972 23396 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:40.513044 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.524555 23441 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:40.539996 18194 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:40.542344 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.543292 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:40.525233 23100 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:40.537667 23312 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.502664 23197 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.527925 23441 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.577876 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:40.580360 18447 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:40.584012 23371 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.580734 18447 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:40.558995 23262 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:18:40.589783 23100 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.592355 23100 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.595669 18445 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:40.598006 23100 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:40.595620 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 80683.1us
I20260709 12:18:40.581002 23217 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.599923 23136 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.616303 23217 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.616776 18194 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 5 to 14, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:40.618738 23217 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.606364 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:40.624210 19040 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:40.609329 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:40.614351 22945 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:40.597544 23460 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:18:40.577490 23312 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:40.615610 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58964.7us
I20260709 12:18:40.618855 18194 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:40.579053 23145 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:18:40.646889 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 130872us
I20260709 12:18:40.647193 19043 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:40.647648 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:40.665853 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:40.643508 23136 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.643760 23227 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:40.660001 18194 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 10 to 12, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 12 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:40.654220 18447 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.655550 23405 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59426 (request call id 1124) took 2841 ms. Trace:
I20260709 12:18:40.672271 23442 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.672490 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:40.681670 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.674610 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 114256us
I20260709 12:18:40.637914 23428 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.653762 23312 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.671165 23427 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:40.692798 18447 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.694466 23420 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:40.676908 23217 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.709211 18375 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:40.710431 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:40.717073 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48437.6us
I20260709 12:18:40.718454 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:40.721375 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73901.4us
I20260709 12:18:40.724423 19040 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:40.733570 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.738131 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:40.681874 23136 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.746781 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.751933 23396 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.753784 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.749658 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:40.759306 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.761075 18300 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:40.728238 23145 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.765401 18445 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 9.
I20260709 12:18:40.762528 18449 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:40.777556 18440 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.779721 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.781149 23414 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.781834 23414 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.783001 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.784022 23414 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.784623 23414 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:40.785437 23414 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:40.786351 23420 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:40.786728 23420 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:40.787699 23420 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.789112 23420 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:40.791460 23100 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:40.792039 23312 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:40.792596 23100 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:40.792989 23100 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:40.793977 23100 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.795894 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.683126 23405 rpcz_store.cc:276] 0709 12:18:37.813952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.859531 (+ 45579us) service_pool.cc:224] Handling call
0709 12:18:38.325940 (+466409us) ts_tablet_manager.cc:934] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
0709 12:18:39.422167 (+1096227us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:40.114557 (+692390us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:40.114788 (+   231us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:40.623651 (+508863us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:40.654774 (+ 31123us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:40.655127 (+   353us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:40.655270 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":4219,"tablet-copy.queue_time_us":455912,"thread_start_us":510825,"threads_started":2}
I20260709 12:18:40.726159 23421 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.772042 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:40.808279 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:40.812449 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.731599 23442 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.817934 19043 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:40.812704 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.819198 18299 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:40.768559 23217 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:40.759008 23197 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:18:40.821836 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:40.822367 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90027.8us
I20260709 12:18:40.795179 23100 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.825655 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.827165 18446 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:40.828117 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.829391 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 77005.6us
W20260709 12:18:40.832244 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52586.2us
I20260709 12:18:40.838053 23472 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:40.838653 23472 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:40.839551 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:40.840171 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:40.843695 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.844239 18375 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:40.824973 23405 tablet_copy_client.cc:806] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:18:40.846060 23405 tablet_copy_client.cc:670] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:18:40.833837 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:40.850920 18375 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:40.835858 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:40.823944 23473 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:40.875830 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59922.8us
I20260709 12:18:40.857405 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:40.858467 23471 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.879253 23471 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:40.879689 23471 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.880774 23442 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.875722 23473 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.883699 23473 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:40.884245 23468 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
I20260709 12:18:40.863874 23441 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.864648 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:40.866386 23414 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.867455 23262 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:40.868321 23421 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.868886 23136 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.836771 23396 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.871022 23417 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:40.873265 22945 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:40.875077 18367 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 12:18:40.877997 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 114937us
I20260709 12:18:40.898522 18447 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:18:40.898927 23476 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:40.901122 18447 pending_rounds.cc:85] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:18:40.854926 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:40.901641 23262 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.902701 18296 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
W20260709 12:18:40.907423 18263 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:18:40.903158 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.903878 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:40.904600 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:40.913915 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65192.5us
I20260709 12:18:40.900650 18367 pending_rounds.cc:85] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 10
I20260709 12:18:40.904923 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.903822 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:40.895349 23414 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:40.903098 23472 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.920254 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.950737 18292 raft_consensus.cc:2410] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:40.951752 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:40.952701 18292 raft_consensus.cc:2410] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:40.921136 18300 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.893909 23420 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:40.921830 18298 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:40.908017 18447 pending_rounds.cc:107] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:18:40.957888 18447 raft_consensus.cc:2889] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.835623 23217 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.923677 18297 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:40.962484 23471 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.923986 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.963335 23478 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:40.924507 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.963938 23478 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:40.924829 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:40.964833 23471 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:40.925431 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:40.965000 23478 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:40.926236 18293 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:40.914853 18367 pending_rounds.cc:107] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:18:40.967967 18367 raft_consensus.cc:2889] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:40.969370 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.927497 23479 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:40.928850 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.971673 23479 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:40.956787 23420 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.932992 18440 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:40.906347 23417 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:40.936233 23414 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.923044 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:40.958699 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 103843us
I20260709 12:18:40.961282 18299 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:40.928524 23136 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:18:40.929376 23262 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:40.978664 18300 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:40.981935 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.982666 18375 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:40.988241 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.989094 18370 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:40.990156 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:40.990474 23217 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:40.990758 18374 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:40.990971 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:40.996130 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73224.3us
I20260709 12:18:40.997484 23227 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:40.997973 18366 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:40.999691 18297 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:40.988241 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.001605 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.001843 23480 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:41.002861 23217 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.003051 23309 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:41.003325 23217 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.003515 23309 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:41.003794 23217 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.004477 23309 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.001791 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:41.006232 23227 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:41.010350 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82351us
I20260709 12:18:41.011492 18374 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:41.014933 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:41.015568 18374 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:41.021611 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.983603 23182 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:41.023043 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:40.973079 23479 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.012974 23309 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:41.026512 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:41.034422 23414 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.035060 23473 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.037551 23473 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.039425 23442 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:41.040047 23442 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.041359 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:41.041132 23442 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.039142 18369 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:41.042470 23417 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.043692 23417 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.045672 18369 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:41.045926 23441 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:41.048782 23262 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.018303 23136 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:18:41.050246 23443 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:41.051090 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.051514 23312 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:41.051910 19040 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:41.051992 23312 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.053946 23420 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.054795 23420 raft_consensus.cc:2764] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:41.054330 23312 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.055833 23312 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:41.021049 22945 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:18:41.066289 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:41.068630 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67141us
I20260709 12:18:41.075280 18374 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:41.067306 23217 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.077611 23471 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.084024 23262 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.087637 23489 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:41.088321 23489 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.091488 23489 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.078729 23458 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:18:41.094563 23471 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.098094 23217 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.087692 23262 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.077883 23480 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:41.111068 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104296us
W20260709 12:18:41.127874 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.240s (SENT) [suppressed 27 similar messages]
I20260709 12:18:41.138638 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.139504 19043 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:41.123143 23471 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.150604 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.151436 18373 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:41.152554 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:41.153475 18371 raft_consensus.cc:2368] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:18:41.159576 23480 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.161474 23262 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:41.150648 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.168053 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:41.172320 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.172940 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:41.181129 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.181491 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.184311 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.184804 18293 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
W20260709 12:18:41.185456 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 349980us
I20260709 12:18:41.139053 23488 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:18:41.191532 18295 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:41.174227 23262 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.193159 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.194324 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.176600 23489 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.198241 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.205832 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.209161 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.183982 23312 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.212489 19043 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:41.195144 23405 tablet_copy_client.cc:538] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:18:41.218181 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.199612 23476 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:41.233433 23500 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.234123 23500 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:41.234959 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.244035 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60605.2us
I20260709 12:18:41.237030 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.238672 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.245391 18447 raft_consensus.cc:2376] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11. Re-sending same reply.
I20260709 12:18:41.239450 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:41.242323 23485 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:18:41.216228 23478 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:41.246207 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.210263 23473 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:41.205296 23489 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:41.228921 23227 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.218849 23312 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:41.266166 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.250181 19038 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:41.209327 23262 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.260010 18296 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:41.293277 23194 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.266280 23417 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.293959 23194 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.294507 23417 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.228299 23420 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:41.296514 23417 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.296703 23194 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.285063 23441 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:41.297288 23312 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.297477 23441 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.298461 23441 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.266220 23489 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.263317 23473 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.300891 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:41.284030 23484 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:18:41.244410 23197 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.322590 23197 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.325500 23197 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.245807 23396 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:41.327064 23396 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.328385 23396 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.266166 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.268719 23227 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.332682 23227 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:41.246080 23500 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.269640 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.270512 23478 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.270742 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.273751 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.280923 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.218706 23309 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:18:41.228065 23479 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:18:41.258740 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.248765 23476 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.286608 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.356983 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.287601 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.288465 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:41.289173 23490 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:18:41.291267 18339 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.571s (SENT)
I20260709 12:18:41.250777 23217 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:18:41.313902 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.314189 23473 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.314709 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.315377 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.316102 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.365113 18443 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:41.365418 18442 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:41.316526 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.316568 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.316128 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.317075 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.366482 18446 raft_consensus.cc:2376] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9. Re-sending same reply.
I20260709 12:18:41.367372 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.321768 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.368108 18449 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:41.368394 23227 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:41.368605 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46945.3us
I20260709 12:18:41.369010 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.369479 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:41.370157 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.333591 19037 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:41.346906 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 73213.7us
I20260709 12:18:41.357792 18374 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:41.356854 19035 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:41.327980 23312 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.363682 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:41.375510 19040 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:41.367377 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.376737 18446 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:41.370283 23227 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:41.252576 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44862.8us
W20260709 12:18:41.378031 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 180539us
I20260709 12:18:41.378688 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.371332 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.372326 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.372788 18447 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:41.364614 18448 raft_consensus.cc:2376] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17. Re-sending same reply.
I20260709 12:18:41.380359 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.381422 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.293314 23262 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.381872 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:41.382103 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 130543us
I20260709 12:18:41.406607 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:41.382227 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.382917 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.383867 18368 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:41.385264 23227 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:41.360584 23414 raft_consensus.cc:2764] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:18:41.359941 23475 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.388011 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.388794 23478 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.371977 23471 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.435911 23309 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.442225 23471 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.392417 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.443214 18443 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:41.399544 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 108346us
I20260709 12:18:41.381923 18448 raft_consensus.cc:2376] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11. Re-sending same reply.
I20260709 12:18:41.420382 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.422888 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.408471 23100 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.423403 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.424573 23479 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.395051 23312 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.428617 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.427116 23414 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:18:41.442296 23309 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:41.423082 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52470us
I20260709 12:18:41.446329 18447 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:41.445107 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.451687 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.427603 23475 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:41.463693 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.459955 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:41.444255 23473 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:41.470999 18443 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:41.471802 23262 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:41.477603 23262 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.478004 23262 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.483405 23476 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.484766 23182 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:41.485569 23479 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:18:41.490180 23505 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:41.491493 23491 raft_consensus.cc:2764] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:18:41.492208 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 113593us
I20260709 12:18:41.453171 23100 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.481247 23227 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.496588 18266 leader_election.cc:400] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:18:41.517503 18266 leader_election.cc:403] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:41.473120 23475 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.421900 23489 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.469237 23136 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:41.446302 23442 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 12:18:41.518086 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56614.4us
W20260709 12:18:41.506547 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 126653us
I20260709 12:18:41.524950 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.526285 23473 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.526767 23473 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:41.527207 23473 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.527374 23503 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.532845 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.533273 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.510942 23479 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:41.538542 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.517196 23100 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.544906 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:41.540130 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 132397us
I20260709 12:18:41.542045 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.551559 18372 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:41.552664 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.553369 18371 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
W20260709 12:18:41.556988 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 105856us
I20260709 12:18:41.532770 23136 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.558501 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.591573 23417 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.592110 23417 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:41.594699 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.594448 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.596992 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:41.512377 23182 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.555188 23194 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.555140 23442 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:41.593546 23262 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.608006 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.609735 18443 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:41.622102 18440 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:41.611591 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 78811.4us
I20260709 12:18:41.617708 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.562773 23136 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:41.596881 23441 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:41.600260 23472 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:41.627842 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 94626.2us
I20260709 12:18:41.632138 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.633883 18445 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:41.605131 23427 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:41.619040 23493 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:41.593156 23417 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:41.640506 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.611888 23194 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.620411 23227 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:18:41.651490 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.651664 18415 leader_election.cc:400] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:41.652542 18415 leader_election.cc:403] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:41.646320 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:41.621286 23309 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:41.618842 23442 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:41.623440 23478 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:41.654526 18448 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:41.666359 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.659713 18446 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:41.670271 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:41.667469 18369 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
W20260709 12:18:41.656988 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 101446us
I20260709 12:18:41.651571 23491 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.681080 18370 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:18:41.683418 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.643337 23427 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:41.643636 23507 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:41.688282 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:41.654213 23194 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.669400 23442 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:41.688781 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:41.700304 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.701479 18369 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:41.702024 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.702690 18374 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:41.704802 18373 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:41.706048 19042 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:41.696017 23427 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.634199 23479 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.738867 18370 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:18:41.700475 23507 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.735570 23489 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.768827 23507 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.770517 23262 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.773422 19041 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:18:41.776469 23182 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.776957 23182 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:41.777390 23182 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.780035 19035 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:41.781333 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.784921 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.785450 19040 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.785827 23502 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:18:41.786289 18449 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:41.738173 23500 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:41.754750 23516 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:41.788194 23182 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:41.755537 23513 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:18:41.789248 23500 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:41.759780 23480 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:41.793032 23480 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:41.762163 23227 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:41.746251 23522 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 9. Current term is 10.
I20260709 12:18:41.787570 23502 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:41.756639 22945 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:18:41.742803 23405 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:18:41.796236 19040 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:41.816804 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:41.740211 23417 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:41.746992 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49262.9us
I20260709 12:18:41.814975 18369 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:41.817657 19041 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.818920 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:41.820936 23489 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:41.821436 23489 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:41.822382 23489 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:41.822669 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 171151us
I20260709 12:18:41.809747 23479 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.822556 23478 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:41.827724 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:41.831465 23473 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:41.835628 18297 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:41.837119 23473 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:41.838438 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:41.839064 18296 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.839661 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.850131 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.851698 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:41.793924 23480 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:41.859727 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:41.863986 18296 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:41.871961 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.884162 18299 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:41.887501 18296 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:41.895059 23476 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:41.905215 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.859953 23522 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:41.905359 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:41.905853 18293 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:41.914419 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:41.917689 23528 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:18:41.921090 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:41.921993 18296 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:41.904742 23473 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.908535 23480 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:41.926743 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:41.867659 23309 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:18:41.904950 23519 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.912012 23522 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:41.944900 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:41.933068 23471 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:41.960201 23309 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:41.960384 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:41.932710 23473 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.963831 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.964933 18295 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:41.977777 23196 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:18:41.978894 23196 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:18:41.983832 23196 ts_tablet_manager.cc:1403] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 1.839s	user 0.051s	sys 0.017s
I20260709 12:18:42.000721 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:41.941871 23519 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:42.014209 18293 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:18:42.014947 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:42.013675 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 329861us
W20260709 12:18:42.017019 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.020747 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:42.020924 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 137903us
I20260709 12:18:41.930301 23182 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 12:18:42.023190 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.997152 23473 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.967650 23309 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:41.999305 23528 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:42.034130 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.020759 23519 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.057811 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.066761 18299 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:42.064891 18293 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:41.948441 23478 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.031996 23528 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.082053 23136 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.082332 23309 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:42.116616 18293 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:42.123263 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:42.123961 18299 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:42.092881 23100 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 12:18:42.082957 23194 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.093425 23521 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:42.098356 23528 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.134085 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:42.145471 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:42.130931 23475 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.002s
I20260709 12:18:42.148864 23475 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.151705 23475 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.146261 18373 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
W20260709 12:18:42.131268 23497 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:42.129827 23100 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.165337 23194 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.177709 18367 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:18:42.188866 23534 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:18:42.161686 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58030.2us
I20260709 12:18:42.127076 23182 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:42.190435 23227 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:42.191016 23533 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:18:42.190585 23182 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.193482 23182 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.193989 23182 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:42.082053 23442 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:42.196965 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 191182us
I20260709 12:18:42.198310 23471 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:42.196731 23442 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:42.199401 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 172582us
W20260709 12:18:42.203835 18339 leader_election.cc:336] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.654s (SENT)
I20260709 12:18:42.134642 23521 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:42.202603 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:42.205142 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.164279 23548 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.187876 23542 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:42.157371 23309 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.214409 19041 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:42.222329 18299 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:42.234229 19035 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:42.216070 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:42.238085 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:42.251036 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.208658 23136 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.201128 23507 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.244994 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:42.228758 23521 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.200976 23504 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:42.270960 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.107637 23522 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.270218 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.274180 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:42.246271 23488 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.239463 22945 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 12:18:42.279479 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 75672.4us
I20260709 12:18:42.284825 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:42.285295 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.285671 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:42.285980 18339 leader_election.cc:336] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.681s (SENT)
I20260709 12:18:42.286547 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.204749 23227 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.250646 23550 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:42.282263 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 77254.7us
I20260709 12:18:42.277590 19037 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:42.199216 23471 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.271270 23443 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:42.334398 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.337798 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.199730 23196 raft_consensus.cc:359] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:42.338750 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51602.1us
I20260709 12:18:42.341576 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:42.345175 18371 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.243726 23548 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.346329 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.282919 23547 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:42.343827 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.345948 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:42.351959 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.343171 23100 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.337297 23500 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:42.355631 23262 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:42.346787 23427 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.002s
I20260709 12:18:42.384276 23427 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.385560 18440 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:42.386077 18440 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:42.386464 18442 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:42.277910 23507 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.357227 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:42.395362 23548 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.395926 23548 raft_consensus.cc:2764] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:18:42.396245 23507 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:42.354162 23479 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.002s
I20260709 12:18:42.397362 19036 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:42.396812 23548 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:42.398754 18448 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:42.399727 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.400147 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.360908 19007 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.400847 23548 consensus_queue.cc:1243] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:42.401530 23548 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.402125 23548 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:42.402751 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.403302 23548 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:42.403695 23548 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:42.404553 23548 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.405663 23548 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.343180 23443 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:42.407269 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.407771 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.408380 18443 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:42.408043 23443 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.409313 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.359671 23500 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.418521 23500 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:42.247273 23542 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.422020 23542 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.422585 23542 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.423431 23539 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:42.356549 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:42.424913 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68428us
I20260709 12:18:42.425062 23472 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:42.429005 23491 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:18:42.429714 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.339537 23553 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:42.363193 23520 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:42.363793 23529 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:42.365749 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.431293 23553 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:42.346096 23196 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:18:42.432686 23196 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:42.377120 23227 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.433688 23553 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.362737 23194 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:42.431298 23520 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.434391 23520 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.438699 23100 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.376973 23519 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.439180 23100 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.441370 23519 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.357414 23547 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.400400 18448 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:42.443023 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.443768 23547 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:42.441797 23100 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.444904 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:42.445660 19040 raft_consensus.cc:606] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:42.445982 19040 raft_consensus.cc:614] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:42.447371 18445 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:18:42.448841 23554 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:18:42.449939 23500 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.394315 23427 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.452111 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:42.456081 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 95268.2us
I20260709 12:18:42.469112 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.289809 23504 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.355341 23488 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:42.471947 18292 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:42.471745 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.470861 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.470541 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.472452 18291 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:42.472050 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.473186 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:42.459776 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 214873us
I20260709 12:18:42.479751 18297 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:42.390878 23262 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:42.488094 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:42.492220 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 88348.4us
I20260709 12:18:42.495509 18447 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:42.496588 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.498027 23196 ts_tablet_manager.cc:1434] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.511s	user 0.008s	sys 0.002s
I20260709 12:18:42.498349 18297 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:18:42.499197 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.502957 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.503787 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.504438 18443 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:42.503691 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:42.506060 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.506367 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:42.505652 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:42.506637 18292 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:42.503564 18295 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:42.480836 23526 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:42.518108 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.498624 23262 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.479643 18440 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.517832 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.520172 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.343679 23550 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.521426 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:42.520946 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:42.525534 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 254479us
I20260709 12:18:42.523635 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:42.499565 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 213612us
I20260709 12:18:42.530413 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.528743 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:42.534261 18300 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:42.534653 18293 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:42.537096 18298 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 12:18:42.538478 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.510715 23427 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:42.541790 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.538987 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.539685 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.534847 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:42.515628 22945 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.540980 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:42.504176 23196 ts_tablet_manager.cc:1467] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Tablet startup took 512ms
I20260709 12:18:42.543206 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.542889 18368 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:42.548483 19044 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:42.560243 18297 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:42.527678 23544 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.529508 23491 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.544227 18440 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.565688 18291 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:42.566061 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.566862 18446 raft_consensus.cc:2368] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:18:42.567058 18291 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:18:42.567759 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:42.567978 23540 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 12:18:42.568406 18291 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:18:42.569650 18291 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:18:42.569895 23227 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.569962 18291 pending_rounds.cc:85] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 9
I20260709 12:18:42.570749 18291 pending_rounds.cc:107] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:42.570581 23227 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.572705 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.574188 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.574805 18449 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:42.575627 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.576143 18449 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.577303 23552 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.572805 23227 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.580229 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.581236 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.582438 18447 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:42.583318 23551 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:42.584110 18448 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:42.584363 23563 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.585594 18448 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:18:42.586839 23549 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:42.586850 18448 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:18:42.569674 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.584448 23500 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.588804 18293 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:42.590324 18447 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:42.592080 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:42.571033 18291 raft_consensus.cc:2889] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:42.594432 18449 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:42.596176 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:42.597009 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.522240 23309 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:18:42.597631 18372 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:42.600348 18264 leader_election.cc:400] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:18:42.593837 19007 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:42.543547 23547 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:42.552435 23480 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:42.643591 18371 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:42.645761 23560 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.652555 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.653472 18372 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:42.652745 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.597098 23471 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:42.596282 23491 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:42.666175 23529 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.667294 23560 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:18:42.667789 23545 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.397s	user 0.000s	sys 0.002s
I20260709 12:18:42.638527 23480 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.670783 23420 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.671339 23479 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:42.672466 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.673779 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:42.674427 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:42.675149 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1181) took 1043 ms. Trace:
I20260709 12:18:42.678470 23471 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:42.785837 23471 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.647053 23478 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:42.682017 18440 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:18:42.793365 18440 pending_rounds.cc:85] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 8
W20260709 12:18:42.793499 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 173715us
W20260709 12:18:42.793676 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 289923us
I20260709 12:18:42.682816 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:42.684413 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:42.724727 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.764705 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:42.767901 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102905us
I20260709 12:18:42.778717 18442 rpcz_store.cc:276] 0709 12:18:41.632152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.682426 (+ 50274us) service_pool.cc:224] Handling call
0709 12:18:41.699404 (+ 16978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.386930 (+687526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.674928 (+287998us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540035fa54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:42.675093 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579541760}
I20260709 12:18:42.795866 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.796242 19035 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:42.797236 18371 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:42.803172 23551 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.803265 23552 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.803741 23551 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.775593 23488 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.804836 23488 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.651628 23262 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.793660 18440 pending_rounds.cc:107] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:18:42.810132 18440 raft_consensus.cc:2889] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:42.807727 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:42.812371 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 269198us
I20260709 12:18:42.779753 23480 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:42.775133 23529 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.822327 23529 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:42.822566 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.823616 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.801162 23560 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.828011 23509 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.218s
I20260709 12:18:42.812245 23478 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:42.832967 23478 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:42.838436 23417 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.012s	user 0.004s	sys 0.005s
W20260709 12:18:42.843833 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49321.2us
I20260709 12:18:42.846253 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.848703 18300 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:42.853618 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.857545 18296 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:42.809610 23100 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.815702 23473 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.001s
I20260709 12:18:42.868561 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:42.875346 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:42.814463 23547 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:18:42.847959 23182 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:18:42.886041 23502 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.886446 23544 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:42.886967 23262 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.887718 23488 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.886128 23100 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:42.861395 23565 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:42.907757 23562 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.884941 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:42.561885 23196 ts_tablet_manager.cc:1469] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Trace:
0709 12:18:33.702744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.703183 (+   439us) service_pool.cc:224] Handling call
0709 12:18:34.054514 (+351331us) ts_tablet_manager.cc:934] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
0709 12:18:35.200497 (+1145983us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:35.819642 (+619145us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:35.819809 (+   167us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:36.072393 (+252584us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:36.072725 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:36.073136 (+   411us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:36.073211 (+    75us) inbound_call.cc:177] Queueing success response
0709 12:18:37.827951 (+1754740us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b1800400d20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65c33ed8 00007f3c65d8ede2 00007f3c6ddf3d23 00007f3c6ddf355d 00007f3c6de8f3d8 00007f3c6dec0f4e 00007f3c6dea5fdd
0709 12:18:38.237487 (+409536us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b4c00280b80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6715a13f 00007f3c6711e2ab 00007f3c6714bf3c 00007f3c6714c461 00007f3c6714c768 00007f3c6ced266b 00007f3c6de0f15b 00007f3c6de0ef98 00007f3c6de00322
0709 12:18:40.145137 (+1907650us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:18:41.987449 (+1842312us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:18:41.988239 (+   790us) tablet_replica.cc:261] Starting instrumentation
0709 12:18:41.988986 (+   747us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":93367,"mutex_wait_us":34983,"raft.queue_time_us":46618,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":12824,"spinlock_wait_cycles":50610688,"tablet-copy.queue_time_us":207616,"tablet-download-pool.queue_time_us":8435,"tablet-download-pool.run_cpu_time_us":33244,"tablet-download-pool.run_wall_time_us":2837429,"thread_start_us":183757,"threads_started":2}
I20260709 12:18:42.908564 23477 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.909047 23477 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.910727 23100 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:42.911460 23527 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:42.911623 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58125.1us
I20260709 12:18:42.912106 23527 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:42.912504 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.912981 23527 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.913678 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:42.869792 23529 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:42.914883 23529 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:42.915956 23529 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:42.845033 23519 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:18:42.910833 23548 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 12:18:42.862111 23500 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:42.883695 23503 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.356s	user 0.013s	sys 0.001s
I20260709 12:18:42.824735 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:42.944124 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:42.944506 18444 raft_consensus.cc:2410] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:18:42.946303 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 122201us
I20260709 12:18:42.951294 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:42.911104 23477 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:42.861159 23523 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.133s
I20260709 12:18:42.965729 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:42.983032 19038 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:42.966275 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:42.966565 23560 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.944864 23479 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.927455 23488 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:42.981305 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:42.981621 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 309215us
I20260709 12:18:42.884930 23473 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:42.958693 23528 consensus_meta.cc:287] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.795s	user 0.008s	sys 0.002s
I20260709 12:18:43.000428 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:42.958433 23503 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:18:43.012494 18375 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:18:42.959800 23547 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:43.014339 19042 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:43.016031 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:42.965196 23548 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:18:42.908355 23562 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.032804 18443 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:43.075433 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 161792us
I20260709 12:18:43.080737 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.082258 23579 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.082634 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.083506 23579 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:43.083878 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.084686 23579 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.085179 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.085958 23579 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:43.086061 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.020131 23560 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.023191 23563 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:43.089843 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.023717 23550 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:43.092478 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.086993 23579 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:43.072525 23571 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:18:43.088171 23542 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.005s
I20260709 12:18:43.018491 23528 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.098690 18372 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:18:43.088936 23504 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.023500 23523 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:18:43.102445 18372 pending_rounds.cc:85] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 10
I20260709 12:18:43.072261 23573 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:43.105316 18372 pending_rounds.cc:107] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:18:43.109658 18372 raft_consensus.cc:2889] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.032471 23488 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:18:43.090232 23563 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.108878 18370 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:43.115754 18446 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:43.116231 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.103528 23542 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:43.101809 23491 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:43.121075 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 85985.2us
W20260709 12:18:43.110766 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110440us
I20260709 12:18:43.111631 23573 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.116175 23563 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.131021 23551 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.121656 23479 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.115489 23580 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.163405 23584 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:43.201567 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.201781 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.202510 18292 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:43.131435 23573 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:43.222293 23521 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.932s	user 0.007s	sys 0.003s
I20260709 12:18:43.222839 23521 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.225052 23521 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:43.204528 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74722.2us
I20260709 12:18:43.204671 23580 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:43.205184 23528 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:43.208612 23555 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:43.208842 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.212016 23562 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.214639 23507 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:43.127538 23547 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:43.215492 23427 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:18:43.266993 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.219806 23551 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.217510 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 2127) took 1291 ms. Trace:
I20260709 12:18:43.268276 18296 rpcz_store.cc:276] 0709 12:18:41.926271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.933977 (+  7706us) service_pool.cc:224] Handling call
0709 12:18:43.217328 (+1283351us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54002378d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:43.217451 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053600000}
I20260709 12:18:43.269002 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:43.270641 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:43.271518 19037 raft_consensus.cc:606] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:43.227084 23473 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:43.258692 23580 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:43.261569 23507 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:43.273094 23584 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:43.290129 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:43.267306 23427 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:43.294360 19039 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:43.266363 23480 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.297360 23584 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:43.297708 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:43.297996 23576 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:18:43.277897 23488 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.315281 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.311004 23587 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:43.350495 19035 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:43.309940 23488 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.354516 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:43.323604 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.355145 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:43.350263 23136 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.360126 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.343922 23597 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:18:43.314927 23507 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.362411 18292 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:43.364871 18291 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:43.351264 23587 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:43.369613 18297 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 12:18:43.381268 23602 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:43.381726 23602 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.381531 23597 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:43.382344 23602 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:43.382576 23597 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:43.383306 23597 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.387655 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.365432 23560 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:43.394022 18291 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.400296 23604 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:43.400712 23604 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:43.401602 23604 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.402884 23604 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.314762 23547 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.410012 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:43.410254 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 112620us
I20260709 12:18:43.351850 23584 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.394410 23529 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:43.421175 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.422792 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.425652 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.423846 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.428892 18297 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:43.366789 23136 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:43.432026 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:43.432593 18372 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.432958 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:43.434278 19036 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:43.435051 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:43.436594 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.433653 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:43.430136 23595 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:18:43.451859 18366 raft_consensus.cc:2368] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:18:43.439460 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.444972 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.452533 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:43.397107 23563 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.428462 23473 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:43.323297 23580 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.455298 19039 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:43.517295 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:43.551620 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:43.551844 18415 leader_election.cc:400] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:18:43.552297 18415 leader_election.cc:403] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:18:43.469707 18372 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:43.552256 18373 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.553994 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:43.560400 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:43.567919 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:43.568114 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 961) took 1055 ms. Trace:
I20260709 12:18:43.589798 18300 rpcz_store.cc:276] 0709 12:18:42.512442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.552261 (+ 39819us) service_pool.cc:224] Handling call
0709 12:18:42.552568 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.849221 (+296653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.567823 (+718602us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540021ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:43.568039 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680494464}
I20260709 12:18:43.448150 23528 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:43.497113 23596 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:18:43.609969 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:43.447073 23553 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:43.597954 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 126413us
I20260709 12:18:43.611059 18445 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.450198 23480 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.617029 18445 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:43.495695 23589 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.619573 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:43.432168 23529 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.635250 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:43.497641 23559 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:43.646772 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.648618 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.676862 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 966) took 1154 ms. Trace:
I20260709 12:18:43.677127 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.620540 19042 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:18:43.652874 18445 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.656095 23472 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:43.657600 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2351) took 1221 ms. Trace:
I20260709 12:18:43.657661 19040 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:43.590622 23473 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.664088 23427 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.568022 23522 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.666023 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:43.567945 23530 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:43.503432 23548 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 12:18:43.668581 18300 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.484148 23587 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.610050 23528 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:43.528736 23602 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.611045 23573 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.673206 23529 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.595108 23542 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:43.543056 23597 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.681607 18444 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:43.673662 23136 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:43.679812 18443 rpcz_store.cc:276] 0709 12:18:42.436198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.511801 (+ 75603us) service_pool.cc:224] Handling call
0709 12:18:43.657378 (+1145577us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b5400185bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:43.657517 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150942464}
I20260709 12:18:43.678956 18445 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.669504 18293 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:43.753036 19044 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:43.755800 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 441316us
I20260709 12:18:43.600240 23598 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.187s
I20260709 12:18:43.803793 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.677374 18295 rpcz_store.cc:276] 0709 12:18:42.513813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.584767 (+ 70954us) service_pool.cc:224] Handling call
0709 12:18:42.585044 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.607910 (+ 22866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.607967 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.621427 (+ 13460us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.864560 (+243133us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:42.931537 (+ 66977us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.213120 (+281583us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b180046fc60. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:43.636825 (+423705us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b54001b29a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df7733 00007f3c68e7cdbf 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c
0709 12:18:43.636906 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.637017 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.637077 (+    60us) raft_consensus.cc:1685] finished
0709 12:18:43.637093 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.668359 (+ 31266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35300,"prepare.queue_time_us":55701,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":12282,"raft.queue_time_us":43356,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":36065,"spinlock_wait_cycles":19775616,"thread_start_us":241493,"threads_started":1,"wal-append.queue_time_us":293882}
I20260709 12:18:43.616886 23553 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:43.621048 23488 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.679680 23472 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:43.864593 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:43.879215 18366 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:43.879962 18371 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:43.865671 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.866612 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:43.881527 19039 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:43.869578 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.874294 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.883839 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.769858 23478 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.883858 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80174.3us
I20260709 12:18:43.811726 18300 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:43.885670 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:43.886258 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.887943 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:43.888298 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.889608 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.862828 23553 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.890590 23420 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.877288 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:43.901661 18372 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:43.892053 18293 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:43.770964 18298 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:18:43.893761 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:43.867808 23520 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.426s	user 0.009s	sys 0.005s
W20260709 12:18:43.904685 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.905601 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.912935 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:43.917881 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 303956us
I20260709 12:18:43.797350 22945 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:18:43.879428 23542 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:43.844002 23442 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:18:43.929279 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:43.930904 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.938555 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.927325 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:43.939329 18443 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:43.935364 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:43.891321 23488 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:43.949028 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:43.916915 23520 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:43.952451 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:43.953343 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:43.900344 23551 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:43.918377 23560 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:43.960812 18443 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:43.963549 18299 raft_consensus.cc:2368] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12.
I20260709 12:18:43.937063 23521 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:43.978950 18445 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:18:43.982460 18445 pending_rounds.cc:85] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 7
I20260709 12:18:43.982710 18445 pending_rounds.cc:107] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:18:43.969800 23500 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.761080 23528 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.982960 18445 raft_consensus.cc:2889] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.959450 23589 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.899596 23478 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.932116 23480 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:18:43.958860 23563 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:43.764144 23522 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.709774 23559 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.764810 23612 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.300s
I20260709 12:18:43.965133 23560 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:43.972051 23548 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:43.931197 23601 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:18:43.982092 23488 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.986770 23579 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.018s	sys 0.050s
I20260709 12:18:44.052489 18373 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:44.066349 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 142841us
I20260709 12:18:44.067464 23473 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:44.057994 23579 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.018s	sys 0.051s
I20260709 12:18:44.068085 23579 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:44.069064 23579 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:43.958619 23472 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.073165 18443 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:44.074122 18445 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:44.053464 23560 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.059016 23610 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:44.092473 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 255861us
W20260709 12:18:44.103789 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 154868us
I20260709 12:18:44.060653 23488 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.105222 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.111361 19037 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:44.112825 19037 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:44.093652 23472 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.115757 23601 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.115839 23548 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.117825 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.057219 23542 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.113777 22945 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:18:44.130625 18294 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:44.135169 18445 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:44.137254 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.141494 18445 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:18:44.142554 19039 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:44.131470 23602 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.096644 23610 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.129695 23597 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.117861 23509 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:18:44.164410 23522 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.139173 23503 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.166688 23503 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.129779 23527 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:18:44.169131 23503 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.144080 23520 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.134719 23519 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.155505 23488 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:44.156426 23615 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:44.157517 23478 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.151489 23622 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:44.146160 23442 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:18:44.177677 23597 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:44.178315 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:44.178908 23610 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.125391 23601 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:44.196882 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54805.3us
I20260709 12:18:44.140405 23473 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:44.150887 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:44.194187 23559 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.196740 18445 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:44.143700 23525 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:44.199600 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.171779 23520 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.139124 23544 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:18:44.207660 23442 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:44.201107 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:44.208806 19039 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.210413 23509 consensus_queue.cc:1243] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:44.211742 23442 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:44.212172 23442 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:44.194824 23597 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.223898 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.225821 23442 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.227198 23442 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.227830 23597 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.213001 23420 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:44.232800 23560 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.238281 23405 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:18:44.240767 18366 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:44.241060 23601 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.241745 23601 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:44.241685 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 90901us
W20260709 12:18:44.204528 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 62750.3us
I20260709 12:18:44.214730 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.203416 18445 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:44.248121 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.248525 23604 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:44.275008 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.274857 23604 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.278546 23627 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.279266 23627 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.282281 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.283696 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.284502 18298 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.242358 23601 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.214375 23542 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.286150 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:44.239149 23530 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:18:44.281499 23405 tablet_bootstrap.cc:492] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:18:44.298753 23405 ts_tablet_manager.cc:1403] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 2.574s	user 0.042s	sys 0.016s
I20260709 12:18:44.277776 23559 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:44.196152 23519 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.305958 23519 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.246136 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.313680 23530 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:44.313886 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 67830.2us
I20260709 12:18:44.314471 23530 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:44.315514 23530 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.316020 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.254382 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:44.246136 19039 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:44.339273 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:44.341885 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55815us
I20260709 12:18:44.330919 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:44.346225 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:44.335808 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.290513 23601 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.281097 23597 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.360584 23597 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.362673 23597 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.280823 23520 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.288129 23420 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.373710 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.365077 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:44.376132 18441 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.360605 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:44.367578 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.343014 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:44.379614 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.392726 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:44.362998 23552 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:44.396941 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50824.8us
I20260709 12:18:44.358736 23553 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:44.413757 23552 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:44.398487 23628 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:44.372740 23579 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:44.415809 23579 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:44.367285 23560 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.399761 23405 raft_consensus.cc:359] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:44.398039 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.390089 23420 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.404348 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.404348 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:44.406298 23631 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:18:44.406996 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.407370 23573 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.407562 23520 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.409269 23507 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.412444 23627 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:44.413491 23605 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.488s	user 0.000s	sys 0.002s
I20260709 12:18:44.414646 23552 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.421017 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:44.449880 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:44.450740 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.450779 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.421388 18298 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:44.425249 18441 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:44.377436 23472 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.392021 23597 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.397100 23522 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.416682 23553 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:44.366236 23542 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.364982 23529 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 12:18:44.447885 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 193620us
I20260709 12:18:44.461289 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.432422 23405 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:18:44.461861 18295 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:44.465159 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.456941 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:44.467777 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.468032 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.455971 23520 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:44.462616 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:44.469913 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:44.462098 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.457093 23503 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.475111 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.475481 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:44.476063 18375 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:44.477210 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2141) took 1110 ms. Trace:
I20260709 12:18:44.480335 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.465065 23552 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.464187 23522 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:44.416700 23579 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.500020 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.497614 18295 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:44.477586 18292 rpcz_store.cc:276] 0709 12:18:43.366561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.366720 (+   159us) service_pool.cc:224] Handling call
0709 12:18:43.391894 (+ 25174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.392311 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.392353 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.405911 (+ 13558us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.516330 (+110419us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:43.516389 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.516470 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.412883 (+896413us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:44.472323 (+ 59440us) raft_consensus.cc:1685] finished
0709 12:18:44.472344 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.477145 (+  4801us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":10721,"spinlock_wait_cycles":20587648,"thread_start_us":109475,"threads_started":1,"wal-append.queue_time_us":453750}
I20260709 12:18:44.458107 23597 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.472750 23612 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:44.523710 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.529215 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:44.534067 18291 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:44.542040 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:44.527490 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3 received from {username='slave'} at 127.0.0.1:43604
W20260709 12:18:44.514843 23569 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.403s	user 0.003s	sys 0.022s
I20260709 12:18:44.516659 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:44.607399 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90803.1us
I20260709 12:18:44.471822 23573 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.496701 18298 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:44.525691 23624 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:18:44.498432 23610 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.520792 23544 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.589774 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.520272 23405 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:44.589741 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.540066 23530 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:44.600633 18295 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:44.525132 23597 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.709600 23442 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:44.599611 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.711302 23530 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.711740 23530 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:44.711869 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:44.712508 18292 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:44.713418 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.496678 23559 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:44.603840 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3 on tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.612540 23560 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:18:44.716560 18294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2518) took 2263 ms (client timeout 2096 ms). Trace:
W20260709 12:18:44.716974 18294 rpcz_store.cc:269] 0709 12:18:42.453035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.453154 (+   119us) service_pool.cc:224] Handling call
0709 12:18:44.716367 (+2263213us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400002154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:44.716473 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3474769664}
I20260709 12:18:44.717316 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.718277 18294 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:44.718571 18444 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:44.719338 23544 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.721484 23544 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.723686 18293 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:44.725431 18374 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1140) took 1619 ms (client timeout 2100 ms). Trace:
I20260709 12:18:44.725744 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:44.725868 18374 rpcz_store.cc:269] 0709 12:18:43.105920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.106067 (+   147us) service_pool.cc:224] Handling call
0709 12:18:44.725230 (+1619163us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400395654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:44.725353 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701263744}
I20260709 12:18:44.726657 18295 raft_consensus.cc:2368] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14.
I20260709 12:18:44.534896 23528 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.727919 18445 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:44.729444 18445 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:44.730670 18445 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:18:44.737754 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1197) took 1172 ms. Trace:
I20260709 12:18:44.738166 18443 rpcz_store.cc:276] 0709 12:18:43.565353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.971169 (+405816us) service_pool.cc:224] Handling call
0709 12:18:43.979766 (+  8597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.075666 (+ 95900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.737532 (+661866us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b54000d8254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:44.737696 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207371136}
I20260709 12:18:44.705464 23522 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.725658 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:44.630926 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:44.528779 23601 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.701793 18298 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:44.595042 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:44.681725 23631 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:18:44.768719 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 173767us
I20260709 12:18:44.768898 19006 leader_election.cc:400] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14.
I20260709 12:18:44.706303 18448 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:44.601910 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:44.744247 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 874) took 1795 ms. Trace:
W20260709 12:18:44.745381 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.765758 18294 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:18:44.770264 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.753939 18292 raft_consensus.cc:2376] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11. Re-sending same reply.
I20260709 12:18:44.769384 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:44.772991 18448 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:44.772964 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.774530 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.770076 19044 rpcz_store.cc:276] 0709 12:18:42.948872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.949016 (+   144us) service_pool.cc:224] Handling call
0709 12:18:43.691670 (+742654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.859543 (+167873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.744056 (+884513us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b5400748bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:44.744172 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1707209472}
W20260709 12:18:44.742722 23609 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.686s	user 0.000s	sys 0.002s
W20260709 12:18:44.743669 23607 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.676s	user 0.000s	sys 0.003s
W20260709 12:18:44.741657 23639 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 12:18:44.775501 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:44.777413 18448 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:44.705859 23573 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:44.707353 23619 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:44.790149 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.740990 23528 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:44.792233 18448 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:44.771946 23587 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:44.778525 23618 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.771535 23598 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.778280 23479 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:44.775949 23560 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.777935 23622 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.751471 23552 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:18:44.794776 23620 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.789592 23542 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.771863 23527 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:18:44.778631 23544 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:18:44.792523 23420 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.796154 23618 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:44.831283 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.838488 23601 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.845347 23618 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.795089 23619 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.830540 23604 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.817308 23622 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.847553 23580 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.797572 23597 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.844302 23530 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.849004 23597 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.849426 23597 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.850586 23405 ts_tablet_manager.cc:1434] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.551s	user 0.010s	sys 0.000s
W20260709 12:18:44.851020 23405 ts_tablet_manager.cc:1467] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Tablet startup took 551ms
I20260709 12:18:44.816569 18292 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:44.856120 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:44.861054 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2404) took 1005 ms. Trace:
I20260709 12:18:44.861590 18371 rpcz_store.cc:276] 0709 12:18:43.856004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.857195 (+  1191us) service_pool.cc:224] Handling call
0709 12:18:43.860075 (+  2880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.880524 (+ 20449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.860763 (+980239us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54000f8a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:44.860976 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45184896}
I20260709 12:18:44.811277 23478 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:44.873277 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.836201 23528 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.850680 23530 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.815646 23479 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:44.877066 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2150) took 1427 ms. Trace:
I20260709 12:18:44.881283 19041 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:18:44.894162 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2527) took 1597 ms. Trace:
I20260709 12:18:44.894667 18296 rpcz_store.cc:276] 0709 12:18:43.297039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.298844 (+  1805us) service_pool.cc:224] Handling call
0709 12:18:43.299179 (+   335us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:43.307462 (+  8283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.307492 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:43.312053 (+  4561us) log.cc:844] Serialized 78 byte log entry
0709 12:18:43.674295 (+362242us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:43.674333 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.674401 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.742082 (+1067681us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:44.888268 (+146186us) raft_consensus.cc:1685] finished
0709 12:18:44.888290 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.894082 (+  5792us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424,"raft.queue_time_us":5450,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3006,"spinlock_wait_cycles":36503936,"thread_start_us":361693,"threads_started":1,"wal-append.queue_time_us":659866}
I20260709 12:18:44.891059 18297 rpcz_store.cc:276] 0709 12:18:43.449851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.451256 (+  1405us) service_pool.cc:224] Handling call
0709 12:18:43.451676 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.452095 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.452135 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.452754 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.885431 (+432677us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:43.885471 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.885544 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.743112 (+857568us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:44.867051 (+123939us) raft_consensus.cc:1685] finished
0709 12:18:44.867071 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.877002 (+  9931us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1368,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":16916224,"thread_start_us":432197,"threads_started":1,"wal-append.queue_time_us":586801}
I20260709 12:18:44.883196 23622 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:44.883728 23573 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:44.851374 23405 ts_tablet_manager.cc:1469] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Trace:
0709 12:18:37.813952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.859531 (+ 45579us) service_pool.cc:224] Handling call
0709 12:18:38.325940 (+466409us) ts_tablet_manager.cc:934] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771)
0709 12:18:39.422167 (+1096227us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:18:40.114557 (+692390us) tablet_metadata.cc:665] Metadata flushed
0709 12:18:40.114788 (+   231us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:18:40.623651 (+508863us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:18:40.654774 (+ 31123us) tablet_replica.cc:206] Creating consensus instance
0709 12:18:40.655127 (+   353us) ts_tablet_manager.cc:989] Replica registered.
0709 12:18:40.655270 (+   143us) inbound_call.cc:177] Queueing success response
0709 12:18:40.941011 (+285741us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b4c00280b80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67148a20 00007f3c67159a29 00007f3c6715a13f 00007f3c6711e2ab 00007f3c6714bf3c 00007f3c6714c461 00007f3c6714c768 00007f3c6ced266b 00007f3c6de0f15b 00007f3c6de0ef98 00007f3c6de00322
0709 12:18:41.719523 (+778512us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:18:44.299137 (+2579614us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:18:44.299807 (+   670us) tablet_replica.cc:261] Starting instrumentation
0709 12:18:44.300488 (+   681us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":4219,"mutex_wait_us":52995,"raft.queue_time_us":10671,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":13842,"spinlock_wait_cycles":17084160,"tablet-copy.queue_time_us":455912,"tablet-download-pool.queue_time_us":197,"tablet-download-pool.run_cpu_time_us":3703,"tablet-download-pool.run_wall_time_us":339448,"thread_start_us":510825,"threads_started":2}
I20260709 12:18:44.887475 23528 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.905179 23619 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:44.889969 23478 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:44.917219 18299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2514) took 2465 ms (client timeout 1991 ms). Trace:
W20260709 12:18:44.921406 18299 rpcz_store.cc:269] 0709 12:18:42.451727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.451881 (+   154us) service_pool.cc:224] Handling call
0709 12:18:44.916944 (+2465063us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54000721d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:44.917130 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3437085568}
I20260709 12:18:44.919293 19037 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:44.922992 19037 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.919926 23479 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:44.953183 23509 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:44.953744 23509 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:44.961114 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.962421 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:44.965541 23503 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.966187 23503 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.966539 23503 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.967134 23503 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.969453 23527 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:44.993626 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 203533us
I20260709 12:18:44.993770 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.999439 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.005529 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.007139 23529 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.008932 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:45.018095 23638 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 12:18:45.019575 19037 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:44.945475 23627 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.024750 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:44.990499 23619 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:45.004303 23626 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.035475 23520 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.946342 23597 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.004030 23562 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.010528 23579 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.030500 23604 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:45.046662 23478 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:44.954794 23509 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.055114 23598 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.055188 23420 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:45.051870 23601 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.033174 23529 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:18:45.074569 23420 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.059082 23620 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.075585 23620 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.076972 23420 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.077538 23420 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:45.077709 23620 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.078294 23620 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.078068 23420 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.065158 23509 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.055961 23619 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.056804 23560 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.083494 23560 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.084954 23619 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.045866 23627 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.085662 23560 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:45.086357 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.086963 23627 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.058795 23473 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:45.087525 23627 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.078756 23620 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.056862 23507 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.093340 23627 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.093755 23507 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.095111 23427 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.096068 23627 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.097543 23597 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.062215 23579 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.101169 23579 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.091918 23473 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.071944 23598 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.105113 23620 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.056273 23626 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:45.123027 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 156975us
I20260709 12:18:45.126547 23420 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.127323 23655 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:18:45.059731 23562 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:45.105115 23598 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.089303 23601 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.119606 23652 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.148367 23652 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.151610 23507 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.151162 23652 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:45.126536 23503 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:45.122054 23620 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:45.160683 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.168244 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:45.171088 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.172781 18371 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:45.176023 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:45.129419 23627 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:18:45.178324 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.178619 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:45.121961 23544 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:18:45.180490 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.179359 18373 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:45.179504 18371 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:45.171908 23507 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:45.184221 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.185462 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.186118 23651 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.186144 18373 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:45.186874 23651 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.178902 23598 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.188035 18373 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:45.187793 23598 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.189602 18369 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:45.190817 23598 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.192543 23562 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.195333 23604 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:45.196130 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.196636 23503 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.196311 23604 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.200896 18375 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:45.146509 23618 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.206697 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.209028 18375 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:45.177910 23528 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:45.197033 23598 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.211562 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.218278 23601 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.218779 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.219460 18375 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 10.
W20260709 12:18:45.220028 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90519.2us
I20260709 12:18:45.173091 23620 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.222625 18371 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 11: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:45.223162 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:45.228016 23626 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.231951 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2658) took 1030 ms. Trace:
I20260709 12:18:45.232352 18447 rpcz_store.cc:276] 0709 12:18:44.201575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.759013 (+557438us) service_pool.cc:224] Handling call
0709 12:18:44.781630 (+ 22617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.912562 (+130932us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:44.912596 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:44.913751 (+  1155us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.057195 (+143444us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:45.057246 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.057348 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.231720 (+174372us) raft_consensus.cc:1685] finished
0709 12:18:45.231740 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.231895 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9491,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":14647,"thread_start_us":253944,"threads_started":2,"wal-append.queue_time_us":212649}
I20260709 12:18:45.234614 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.236754 23655 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:45.250164 23627 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:45.251937 23544 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:45.252463 23544 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:45.256162 23618 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.253515 23544 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.264209 23629 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:45.264565 23620 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.264838 23629 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:45.222821 23598 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.266002 23629 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.269814 18370 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.272708 23629 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.277272 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.272848 23598 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:45.280129 23611 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.272s	user 0.000s	sys 0.002s
I20260709 12:18:45.279803 23598 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:45.281256 23598 consensus_queue.cc:1243] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:45.283159 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:45.283903 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 159994us
I20260709 12:18:45.285776 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:45.290841 23527 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:45.290447 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:45.290920 23652 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:45.291388 23527 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.291817 23527 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.291865 23652 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:45.292371 23652 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:45.293617 23539 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.294401 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:45.290012 23227 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:45.295044 23227 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.295385 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.295626 18447 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:45.296166 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.296833 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.297016 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.297485 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.296207 18444 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:45.298635 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.298727 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.299662 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.299849 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.294150 23539 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.301173 18442 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:45.301134 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.302114 18446 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:45.302443 18449 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:45.295456 23227 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.297426 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:45.293304 23652 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.292289 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:45.308310 22945 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:45.308539 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.308785 22945 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.309240 22945 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.309391 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.310026 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:45.311582 18440 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:18:45.312410 23550 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.313020 23652 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.312891 23550 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.314054 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:45.314582 18367 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.315495 23539 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:45.321094 18366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2413) took 1427 ms (client timeout 1641 ms). Trace:
W20260709 12:18:45.321473 18366 rpcz_store.cc:269] 0709 12:18:43.893329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.893482 (+   153us) service_pool.cc:224] Handling call
0709 12:18:45.320933 (+1427451us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000c9ed4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:45.321035 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775433984}
I20260709 12:18:45.327808 18369 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:18:45.327862 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.328444 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.329736 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.330467 18374 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:45.327275 23668 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:18:45.332391 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:45.333004 18366 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:45.334033 23539 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:45.332533 18443 pending_rounds.cc:85] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 5
I20260709 12:18:45.335481 18443 pending_rounds.cc:107] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:45.335747 18443 raft_consensus.cc:2889] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:45.336488 23550 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:45.340397 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.346736 18367 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:45.347467 18371 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:45.348986 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2374) took 1787 ms. Trace:
I20260709 12:18:45.349323 18440 rpcz_store.cc:276] 0709 12:18:43.562030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.562155 (+   125us) service_pool.cc:224] Handling call
0709 12:18:43.607833 (+ 45678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.311965 (+1704132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.348744 (+ 36779us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54000d9b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:45.348908 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3917426048}
W20260709 12:18:45.358295 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.364948 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.365664 18440 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:45.373734 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:45.376163 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:45.376987 19042 raft_consensus.cc:606] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:45.356750 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:45.343021 23544 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.392138 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.394109 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:45.396687 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:45.397073 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 68716.3us
I20260709 12:18:45.397420 18446 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:45.399132 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.389202 23530 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:45.404139 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.406119 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.405382 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:45.389029 23670 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:18:45.411597 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:45.408557 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.349551 23227 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.427933 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.428524 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:45.393673 23610 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.431221 18446 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:45.450656 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.451511 19037 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:45.377565 23442 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:18:45.404171 23601 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.470149 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.472522 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.410780 23587 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.410631 23525 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:18:45.458683 18194 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 213) took 21509 ms (client timeout 15000 ms). Trace:
I20260709 12:18:45.458775 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.435859 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:45.463074 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:45.410980 23539 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.410075 23664 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:45.485855 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50059.4us
I20260709 12:18:45.486115 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:45.486944 19044 raft_consensus.cc:606] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:45.487330 19044 raft_consensus.cc:614] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:45.426218 23550 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.426523 23622 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.471180 23442 consensus_queue.cc:1243] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:18:45.442927 23542 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:45.413097 23668 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:18:45.478040 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.462981 23628 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:45.443625 23520 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:45.431600 23667 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 12:18:45.504350 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 479665us
I20260709 12:18:45.442790 23136 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:45.489298 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.492815 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.495877 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.494467 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:45.453536 23610 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:45.482771 23626 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:45.479674 23525 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.442500 23604 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:18:45.513573 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:45.480240 18194 rpcz_store.cc:269] 0709 12:18:23.934167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.934422 (+   255us) service_pool.cc:224] Handling call
0709 12:18:45.440861 (+21506439us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:40.968632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:40.969288 (+   656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:40.969370 (+    82us) write_op.cc:435] Acquired schema lock
0709 12:18:40.969394 (+    24us) tablet.cc:662] Decoding operations
0709 12:18:40.970711 (+  1317us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:40.970804 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:40.970837 (+    33us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:18:40.972626 (+  1789us) tablet.cc:701] Row locks acquired
0709 12:18:40.972670 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:18:40.972889 (+   219us) write_op.cc:270] Start()
0709 12:18:40.973068 (+   179us) write_op.cc:276] Timestamp: P: 1783599520972809 usec, L: 0
0709 12:18:40.973104 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:18:41.039544 (+ 66440us) log.cc:844] Serialized 3601 byte log entry
0709 12:18:41.631341 (+591797us) op_driver.cc:464] REPLICATION: finished
0709 12:18:41.888604 (+257263us) write_op.cc:301] APPLY: starting
0709 12:18:41.888727 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:41.897723 (+  8996us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:41.897879 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:42.371703 (+473824us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:42.372080 (+   377us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:18:42.372136 (+    56us) write_op.cc:312] APPLY: finished
0709 12:18:42.372560 (+   424us) log.cc:844] Serialized 89 byte log entry
0709 12:18:42.372781 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:42.372954 (+   173us) write_op.cc:454] Released schema lock
0709 12:18:42.373024 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:42.373542 (+   518us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:42.373577 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3434347,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":243727,"apply.run_cpu_time_us":19346,"apply.run_wall_time_us":484931,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41711,"num_ops":9,"prepare.queue_time_us":101912,"prepare.run_cpu_time_us":17115,"prepare.run_wall_time_us":172162,"raft.queue_time_us":138988,"raft.run_cpu_time_us":24999,"raft.run_wall_time_us":360938,"replication_time_us":627089,"thread_start_us":457909,"threads_started":4,"wal-append.queue_time_us":159730,"wal-append.run_cpu_time_us":20740,"wal-append.run_wall_time_us":2168764}]]}
I20260709 12:18:45.493607 23598 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:45.477515 23601 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.519321 19039 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:45.517552 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.523435 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.487377 23539 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.498126 23520 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.524909 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:45.489343 23562 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:18:45.534351 19038 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:45.499663 23618 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:18:45.497354 23622 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.540197 18445 raft_consensus.cc:2410] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:18:45.490394 23664 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:45.509990 22945 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.546216 23678 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.528164 23550 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.559326 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:45.553905 23227 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.517465 23529 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:18:45.562027 18294 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1f155085cb647ff957b468384227872"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:18:45.542665 23679 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.562767 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2534) took 1097 ms. Trace:
I20260709 12:18:45.565014 23651 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:45.566596 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 522050us
W20260709 12:18:45.567505 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 73106.7us
I20260709 12:18:45.772445 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.580991 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 979) took 1040 ms. Trace:
W20260709 12:18:45.607987 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 84619.9us
I20260709 12:18:45.774812 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:45.771492 23227 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:45.776643 23227 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.777271 23622 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.778777 23539 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.779026 23473 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:45.779305 23227 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:45.635659 22945 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.621912 18300 rpcz_store.cc:276] 0709 12:18:44.465844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.466113 (+   269us) service_pool.cc:224] Handling call
0709 12:18:44.468679 (+  2566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.468876 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:44.702668 (+233792us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.030914 (+328246us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.278345 (+247431us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b18004c6300. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:45.414239 (+135894us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b18004c6300. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:45.562447 (+148208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:45.562497 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:45.562550 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.562611 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.562716 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":386842,"prepare.queue_time_us":406178,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":401234,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":11923,"spinlock_wait_cycles":38718080,"thread_start_us":681532,"threads_started":3,"wal-append.queue_time_us":361222}
I20260709 12:18:45.621770 23529 consensus_queue.cc:1243] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:45.596002 23598 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:45.581370 23525 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:45.575456 23527 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.581621 23604 consensus_queue.cc:1243] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:45.580875 23610 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:45.619331 23678 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.608911 23520 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.646534 23679 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:45.578673 23673 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:18:45.614233 23587 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:45.627020 23550 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:45.619498 23676 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:45.568379 23580 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:45.611370 23662 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:45.768195 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 138028us
I20260709 12:18:45.619920 18294 raft_consensus.cc:606] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:45.568185 19037 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:45.570551 23442 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.616645 19041 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 11: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:45.569020 23675 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.782032 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.773200 18298 rpcz_store.cc:276] 0709 12:18:44.540358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.763844 (+223486us) service_pool.cc:224] Handling call
0709 12:18:44.764211 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.764608 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.764647 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:45.261973 (+497326us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b2c000cd878. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:18:45.262551 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.407082 (+144531us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:18:45.426415 (+ 19333us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.457585 (+ 31170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.457662 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.569004 (+111342us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b1800445cc0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:45.580775 (+ 11771us) raft_consensus.cc:1685] finished
0709 12:18:45.580797 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.580926 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71482,"prepare.queue_time_us":18742,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":36742,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":7941,"spinlock_wait_cycles":22013568,"thread_start_us":144100,"threads_started":1,"wal-append.queue_time_us":236970}
I20260709 12:18:45.641417 23542 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:45.794556 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 269690us
I20260709 12:18:45.783797 23667 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.967934 18294 raft_consensus.cc:614] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:45.787209 23598 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.784189 23530 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:45.971159 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.986979 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:45.971887 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:45.968750 23658 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:18:45.845786 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 71040.5us
I20260709 12:18:45.783836 23478 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:45.973484 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:45.988325 22945 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:45.856875 23629 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.583s	user 0.016s	sys 0.002s
I20260709 12:18:45.999864 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:46.002048 22945 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.811695 23618 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:45.992914 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:45.882056 23663 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:46.004640 22945 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:45.889106 23227 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.872265 23668 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:46.007637 18444 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.970511 23580 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.020257 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:46.012079 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:45.872773 23659 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:18:46.033772 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 423357us
W20260709 12:18:45.990778 23620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.003s	sys 0.029s
I20260709 12:18:46.042665 22945 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:45.852298 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 65218.4us
I20260709 12:18:46.010700 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:46.046327 19039 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:45.969421 23442 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:46.014882 23601 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.016608 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:45.871812 18195 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 6 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 13 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:45.965651 23676 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:45.975723 23691 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:45.992550 23674 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:46.067708 18195 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:45.817749 23684 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.202s
I20260709 12:18:46.033412 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:46.033823 18367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.054s	sys 0.083s
W20260709 12:18:46.075707 18367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.054s	sys 0.083s
I20260709 12:18:46.010177 23503 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:45.836455 23604 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.889626 23541 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.137s
I20260709 12:18:46.040264 19035 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:46.014045 23663 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:45.985051 23528 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.013926 23519 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:46.103695 23519 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:46.104264 23519 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:45.975654 23675 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.107919 23550 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:46.075829 23691 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.109414 23675 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.051716 23693 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.056303 23587 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.111552 23562 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:46.056458 23442 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.019181 23680 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:46.057482 23530 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.058283 23542 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:46.052066 23136 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:18:46.012229 23478 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.033218 23580 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.018169 23668 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:46.019683 23651 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:46.034158 23659 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:46.044086 23620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.004s	sys 0.029s
I20260709 12:18:46.064770 23629 raft_consensus.cc:993] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:45.990514 23667 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.055521 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:46.020311 23664 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.078213 22945 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.077446 18195 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 10, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 10 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:46.016095 23529 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:18:46.083000 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49665.2us
W20260709 12:18:46.065443 23522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.066s
I20260709 12:18:46.020869 23227 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.069329 23509 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.069164 23674 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:46.087930 23601 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:46.091723 23544 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.688s	user 0.016s	sys 0.000s
W20260709 12:18:46.139945 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 84506us
I20260709 12:18:46.087803 23604 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:46.140311 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 367903us
I20260709 12:18:46.145807 19035 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:46.148236 22945 raft_consensus.cc:1081] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:46.169468 18195 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 6 to 9, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 9 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:46.147514 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 141659us
I20260709 12:18:46.290488 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:46.111578 23693 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.136412 23680 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.111210 23691 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:46.181883 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:46.178629 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.184558 19044 pending_rounds.cc:85] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 8
I20260709 12:18:46.298769 19044 pending_rounds.cc:107] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 13.9
W20260709 12:18:46.186192 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 133621us
W20260709 12:18:46.147879 23522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.066s
I20260709 12:18:46.186975 18369 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:46.303963 18369 pending_rounds.cc:85] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:18:46.304200 18369 pending_rounds.cc:107] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:46.304407 18369 raft_consensus.cc:2889] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:46.121266 23676 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:46.248446 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:46.318132 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 126820us
I20260709 12:18:46.318854 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:46.143772 23550 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:46.111156 23598 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.185658 23674 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.250290 19036 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:46.183214 23628 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:46.179913 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.121551 23528 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:46.186501 23679 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:46.249276 23604 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:46.284713 23685 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:18:46.299171 19044 raft_consensus.cc:2889] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:46.249465 23529 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:46.180930 23664 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 12:18:46.309935 23682 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:46.336619 23696 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:46.311645 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:46.312995 23681 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.313751 23688 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:46.147269 23442 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.334820 23697 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:18:46.324366 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 136115us
I20260709 12:18:46.315462 18195 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 8 to 11, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:46.182726 23663 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.303414 23680 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.331470 23550 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:46.339754 23698 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:46.368006 23618 consensus_queue.cc:1175] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" responder_term: 12 status { last_received { term: 10 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:18:46.351459 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.370635 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:46.400619 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:46.402647 18443 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.361238 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:46.405185 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44043.9us
I20260709 12:18:46.407145 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:46.408169 23627 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.408746 23627 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.408983 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:46.409796 18441 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:46.368623 23509 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.412066 23509 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.412118 23703 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:18:46.413460 23664 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:46.414347 23626 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:46.414748 23626 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:46.415786 23626 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.416832 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:46.417832 18441 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:46.418810 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:46.424929 18445 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:46.413470 23672 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.894s	user 0.003s	sys 0.000s
I20260709 12:18:46.419389 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:46.446101 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 134584us
I20260709 12:18:46.446538 18448 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.420928 23503 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:46.454488 18444 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:46.426693 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:46.378667 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 334360us
W20260709 12:18:46.359032 23525 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.001s
I20260709 12:18:46.359942 23136 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.443082 18443 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
W20260709 12:18:46.443950 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.262419 23668 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.451854 22945 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:46.373939 23527 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.424922 18441 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 9.
I20260709 12:18:46.346266 23697 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:46.458505 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 87993.1us
I20260709 12:18:46.351262 23678 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:46.459105 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:46.402670 23676 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.411015 23627 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.286186 23659 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.529696 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:46.473942 18366 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:18:46.355676 23550 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.531725 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.476588 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:46.344707 23696 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.353983 18195 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 6 to 8, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 8 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:46.480732 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:46.480855 18374 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:46.507332 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:46.472400 18449 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:46956
I20260709 12:18:46.361088 23472 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 12:18:46.460777 23539 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.454730 23503 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:46.342758 23529 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.476439 23629 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.535068 18449 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:46.617559 18449 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.482493 23662 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:46.482654 23601 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:46.567628 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.609045 18195 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 5 to 10, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 10 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:46.600286 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.652671 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.533986 23668 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.345657 23688 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.534358 19041 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:46.545284 18448 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:46.535063 19039 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:46.615764 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.345407 23681 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.616843 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.538987 23667 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.366773 23680 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.475742 23473 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:46.618636 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.537153 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 237189us
I20260709 12:18:46.618929 18445 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:46.459698 23604 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.620049 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1156) took 1292 ms. Trace:
I20260709 12:18:46.668138 18375 rpcz_store.cc:276] 0709 12:18:45.327765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.328458 (+   693us) service_pool.cc:224] Handling call
0709 12:18:45.328722 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.329092 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:45.329120 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:45.333776 (+  4656us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.434421 (+100645us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:45.434490 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.434581 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.412861 (+978280us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:46.612928 (+200067us) raft_consensus.cc:1685] finished
0709 12:18:46.612951 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.619968 (+  7017us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":11018112,"thread_start_us":100214,"threads_started":1,"wal-append.queue_time_us":154239}
I20260709 12:18:46.340543 23628 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.511368 23651 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.610706 18374 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:46.514374 23697 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:46.692456 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.425689 23698 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:46.659124 18371 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:46.660280 18366 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 9.
I20260709 12:18:46.481707 23682 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.608645 23704 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:46.612884 23696 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.616639 23659 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.616463 23700 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.663851 18195 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:46.649233 23629 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.623827 23662 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:46.652864 23699 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:18:46.699756 18374 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:18:46.706275 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.658010 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 288109us
W20260709 12:18:46.707373 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 416965us
I20260709 12:18:46.709945 18195 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:46.471315 23525 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.708201 18370 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:46.681162 18449 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.726205 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:46.700975 23227 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:46.691797 23473 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:46.738261 23701 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:18:46.739111 23473 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:46.701857 23503 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.709189 23700 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.740356 18449 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:46.710772 23598 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.713075 23693 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:46.722641 23662 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.747040 19040 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:46.700649 23651 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.735671 23539 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.806156 23610 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:46.806401 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 81226.3us
I20260709 12:18:46.786779 18195 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 10 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.809763 18195 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 8, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 8 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:46.813536 18195 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:46.815352 18195 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 9, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 9 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:46.820201 18195 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 8 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:46.817771 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.821033 23509 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.806870 23610 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.827720 23509 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.711305 23682 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.847886 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.845737 23472 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:46.846113 23698 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.848664 18374 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:46.711916 23704 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:46.829986 23675 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:46.856287 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119232us
I20260709 12:18:46.863274 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:46.868925 18371 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.868171 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.871213 18374 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:46.856750 19039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:46.849682 23702 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
W20260709 12:18:46.853902 23580 consensus_meta.cc:287] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.002s
I20260709 12:18:46.883450 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:46.888885 18375 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
W20260709 12:18:46.846858 23705 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 12:18:46.755450 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 273483us
I20260709 12:18:46.883875 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:46.885984 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.864679 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54465.6us
I20260709 12:18:46.841786 23539 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:46.850507 23472 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:46.890108 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:47.144187 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1133) took 1632 ms. Trace:
I20260709 12:18:46.940335 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:47.145648 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1203) took 1301 ms. Trace:
I20260709 12:18:47.005715 18446 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:18:47.167397 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.176235 18368 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:46.945739 19039 raft_consensus.cc:606] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:46.938061 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:47.177197 19039 raft_consensus.cc:614] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:47.178452 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:46.989449 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.179674 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.177873 18374 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:46.936021 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52210.3us
I20260709 12:18:47.179831 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:47.174120 18375 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.184716 18442 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:46.905189 18371 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:46.922430 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1126) took 1494 ms. Trace:
I20260709 12:18:47.181058 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:47.169145 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.186683 19036 rpcz_store.cc:276] 0709 12:18:45.427639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.427854 (+   215us) service_pool.cc:224] Handling call
0709 12:18:46.922166 (+1494312us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b5400834fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:46.922339 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768491776}
I20260709 12:18:47.188721 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.174197 19040 rpcz_store.cc:276] 0709 12:18:45.511754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.511944 (+   190us) service_pool.cc:224] Handling call
0709 12:18:47.143930 (+1631986us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540048ec54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:47.144108 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630737664}
W20260709 12:18:46.936856 23711 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:18:46.943646 23674 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
I20260709 12:18:47.197695 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:46.932762 23677 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.647s	user 0.002s	sys 0.001s
W20260709 12:18:47.006961 23520 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.169s	user 0.009s	sys 0.000s
W20260709 12:18:46.930203 23519 consensus_meta.cc:287] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.825s	user 0.009s	sys 0.002s
I20260709 12:18:46.888712 23528 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:47.176378 23704 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:47.300402 23550 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.689s	user 0.002s	sys 0.008s
I20260709 12:18:46.935686 23529 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.320576 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.320884 23727 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.196471 23707 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.322216 23727 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.322355 23707 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.322887 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:47.324156 23601 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.667s	user 0.003s	sys 0.010s
I20260709 12:18:47.324555 23707 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.324765 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.243131 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:47.325322 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 127729us
I20260709 12:18:47.175472 19038 rpcz_store.cc:276] 0709 12:18:45.844026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.844162 (+   136us) service_pool.cc:224] Handling call
0709 12:18:45.965717 (+121555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.970788 (+  5071us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:45.970822 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:45.988931 (+ 18109us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.260636 (+271705us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:46.260692 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.260773 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.740029 (+479256us) raft_consensus.cc:1685] finished
0709 12:18:46.740051 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.145409 (+405358us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540071e8d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:47.145592 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4595,"raft.queue_time_us":5167,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":14030,"spinlock_wait_cycles":251758976,"thread_start_us":239971,"threads_started":1,"wal-append.queue_time_us":459272}
I20260709 12:18:47.326627 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:47.244534 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52772.5us
W20260709 12:18:47.202133 18338 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.553s (SENT) [suppressed 13 similar messages]
I20260709 12:18:47.327505 18300 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:47.329038 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.330715 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.331432 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:47.333587 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:47.334357 18300 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.336436 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.337111 18291 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:47.338645 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.339291 18291 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:47.340590 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:47.247166 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.341892 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.177700 23472 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.363201 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:46.942703 23626 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.979386 23684 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:47.011713 23525 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.080564 23604 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:46.937669 23629 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:46.930333 23610 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:47.244411 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.324709 23727 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.366072 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:47.132691 23478 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.011436 23651 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.092919 23655 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:46.890148 23716 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:47.365173 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:47.392189 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.192539 23541 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.177152 23700 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.189679 23530 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:47.244117 23136 consensus_meta.cc:287] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.728s	user 0.008s	sys 0.005s
I20260709 12:18:46.883716 23675 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:47.371650 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 102525us
I20260709 12:18:47.374161 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:47.197229 23562 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.324592 23601 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:47.100915 23697 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.380470 18445 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:47.396147 23696 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.000s
W20260709 12:18:47.382156 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 180039us
I20260709 12:18:46.884191 23715 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:47.385434 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190615us
I20260709 12:18:47.385584 18447 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.308671 23703 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:47.305176 23559 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.308928 23708 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:47.376880 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.432255 18293 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:47.365470 23509 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:47.196583 23694 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:47.393362 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:47.192802 23618 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.184s
I20260709 12:18:47.394551 18375 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:47.265537 23473 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.397567 18300 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:47.459458 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.321676 23550 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.312578 23679 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:18:47.408264 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:47.459959 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2160) took 2092 ms. Trace:
I20260709 12:18:47.411424 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.320927 23662 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.408403 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.403126 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:47.378196 23628 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:47.375327 23684 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:47.380954 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.488788 18293 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:47.416661 19039 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:46.896392 23580 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.411468 18297 raft_consensus.cc:2376] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13. Re-sending same reply.
I20260709 12:18:47.491516 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:47.492959 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.421118 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:47.362123 23520 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.364907 23519 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.423964 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1212) took 1444 ms. Trace:
I20260709 12:18:47.443212 23663 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.389551 23655 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:47.362823 23674 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.433754 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:47.476948 23721 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:47.524183 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90494.1us
I20260709 12:18:47.443511 18449 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:47.458438 23509 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:47.447528 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:47.413995 23136 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:47.453264 18445 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:47.477305 18367 rpcz_store.cc:276] 0709 12:18:45.367965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.368144 (+   179us) service_pool.cc:224] Handling call
0709 12:18:45.368545 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.373009 (+  4464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:45.373041 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:45.390031 (+ 16990us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.076169 (+686138us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:46.076231 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.076332 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.932127 (+855795us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:47.433160 (+501033us) raft_consensus.cc:1685] finished
0709 12:18:47.433181 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.459898 (+ 26717us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9663,"raft.queue_time_us":6725,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9741,"spinlock_wait_cycles":5684480,"thread_start_us":646813,"threads_started":1,"wal-append.queue_time_us":730736}
I20260709 12:18:47.431854 23703 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.485339 18292 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
W20260709 12:18:47.562439 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 110511us
I20260709 12:18:47.432936 23503 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.494221 18374 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 11: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:47.497953 18310 tablet_copy_service.cc:342] P e23c0ce8d89940da8eac6a6002e349c5: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8903d70080d24236be955433a92791c7 received from {username='slave'} at 127.0.0.1:43604
I20260709 12:18:47.497831 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.578543 18310 tablet_copy_service.cc:434] P e23c0ce8d89940da8eac6a6002e349c5: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-8903d70080d24236be955433a92791c7 on tablet 8903d70080d24236be955433a92791c7 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:47.426240 18338 leader_election.cc:336] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 2.096s (SENT)
W20260709 12:18:47.514891 23668 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.854s	user 0.011s	sys 0.000s
I20260709 12:18:47.418135 23562 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.511788 18447 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:47.502274 18446 rpcz_store.cc:276] 0709 12:18:45.979417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.979545 (+   128us) service_pool.cc:224] Handling call
0709 12:18:46.006592 (+ 27047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:47.175714 (+1169122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.423728 (+248014us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54000d8254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:47.423908 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271829760}
I20260709 12:18:47.415973 23530 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.517921 18443 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:47.519743 18366 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:47.445178 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:47.525671 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:47.596144 18297 raft_consensus.cc:2368] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14.
I20260709 12:18:47.596936 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.597891 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.598762 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.599548 18297 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:18:47.600206 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.601150 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.601663 18297 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:47.601969 18295 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:47.407599 23716 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.602388 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:47.502300 23676 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.963s	user 0.010s	sys 0.001s
I20260709 12:18:47.413837 23541 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.476753 23722 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:47.479216 23694 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:47.557881 18294 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:47.559074 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.686076 18293 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:47.428777 23696 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.565560 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:47.689911 18442 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
W20260709 12:18:47.690552 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 125048us
W20260709 12:18:47.503713 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 100652us
W20260709 12:18:47.579625 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 153412us
I20260709 12:18:47.581763 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.691854 18299 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:47.692278 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:47.436142 23559 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.587666 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.431048 23715 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.524607 23628 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.515453 23737 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:47.626055 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.485736 23473 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.657992 19041 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:18:47.703039 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.557334 23651 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.442819 23708 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.581211 23552 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:47.681967 18300 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:18:47.687543 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 223568us
I20260709 12:18:47.593923 23539 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:47.699316 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.715866 18299 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:47.691125 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:47.701200 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.560227 23734 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:47.711609 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:18:47.706694 18374 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:47.708455 22945 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:47.582033 23627 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 12:18:47.676820 23542 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:47.710400 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:47.695997 23618 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:47.720360 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:47.720747 18297 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:47.709240 23679 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.720705 18368 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:47.839995 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.818109 18298 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:47.812305 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:47.685457 23610 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:47.815820 18441 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:47.710778 23704 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.719342 23734 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:47.859992 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.717072 23668 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:47.865813 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.869508 18299 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:47.688994 23684 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.813539 23539 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:47.698104 23722 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.715898 23651 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.580160 23721 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.711297 23628 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:47.853009 23679 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.699466 23655 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.733841 23552 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.891837 18369 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:47.894573 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:47.952032 18293 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:47.868525 23732 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:47.872782 23734 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.969846 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.858078 23618 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:47.956161 23743 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:18:47.957207 23740 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 12:18:47.958570 23726 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:18:47.959954 23730 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 12:18:47.960880 23745 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:18:47.871433 23668 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:47.896936 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 186598us
I20260709 12:18:47.962191 23732 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:47.971163 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.717360 23509 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.716298 23737 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.852327 23542 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.983793 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:47.890151 23697 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:47.953841 23679 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.012331 23427 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.056910 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1205) took 1911 ms. Trace:
I20260709 12:18:48.054978 18299 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:48.062677 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:48.063908 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:48.006502 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:48.008678 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:48.020422 23668 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.009894 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.867583 23610 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.011258 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.011693 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:48.011871 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:47.871042 23704 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:47.917785 18441 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:18:47.980074 23655 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.039691 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:48.070185 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 63795.5us
I20260709 12:18:48.070492 19041 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:48.041913 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.071450 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.044885 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.072335 18449 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:48.073271 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:48.051923 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:47.896297 23539 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:48.005549 23675 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:47.985867 23681 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:47.954180 23550 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.055338 23697 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:48.073397 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:48.055361 23674 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.075042 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:48.054154 23659 consensus_meta.cc:287] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.342s	user 0.008s	sys 0.009s
I20260709 12:18:48.044234 23732 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.062107 19044 rpcz_store.cc:276] 0709 12:18:46.145485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.146459 (+   974us) service_pool.cc:224] Handling call
0709 12:18:46.146787 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:46.339653 (+192866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.339688 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:46.342233 (+  2545us) log.cc:844] Serialized 52 byte log entry
0709 12:18:47.100394 (+758161us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b18004b3c40. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:47.424254 (+323860us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:47.424297 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.516496 (+ 92199us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b5400466754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:47.516569 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:48.056689 (+540120us) raft_consensus.cc:1685] finished
0709 12:18:48.056711 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.056853 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":855,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":107092096,"thread_start_us":322974,"threads_started":1,"wal-append.queue_time_us":1181127}
I20260709 12:18:48.081949 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:48.046087 23716 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.046329 23693 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:48.083509 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 240105us
I20260709 12:18:47.957774 23733 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.419s
I20260709 12:18:48.047004 23703 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.112330 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.055254 23680 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:47.958930 23710 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.345s
I20260709 12:18:48.117053 18295 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:47.877684 23694 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.062997 23530 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.120628 19038 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:18:48.156055 19038 pending_rounds.cc:85] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 10
I20260709 12:18:48.156320 19038 pending_rounds.cc:107] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:18:48.156615 19038 raft_consensus.cc:2889] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:48.120429 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49062.5us
I20260709 12:18:48.122926 19036 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:18:48.066430 23562 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.160063 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:48.112339 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 75413.1us
I20260709 12:18:48.095273 23690 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.126034 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:48.003146 23587 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.060115 23749 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 12:18:48.116906 23681 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:48.155194 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.161127 18295 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:48.161679 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:48.163555 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.147634 23716 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:48.082162 23427 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:48.125782 23550 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:48.150527 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53833.2us
I20260709 12:18:48.121486 23541 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:48.170660 18412 connection.cc:256] RPC call timeout handler was delayed by -1.224809771000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 12:18:48.146863 23659 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.126183 23697 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:48.172147 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:48.158036 19036 pending_rounds.cc:85] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 10
I20260709 12:18:48.271544 19036 pending_rounds.cc:107] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:18:48.271915 19036 raft_consensus.cc:2889] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.174146 23628 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.174291 18449 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:18:48.175582 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:48.178148 23626 consensus_meta.cc:287] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.000s
W20260709 12:18:48.178774 23472 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.000s
I20260709 12:18:48.179344 19043 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 10 LEARNER]: Advancing to term 11
I20260709 12:18:48.179615 23688 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.167225 23708 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.174103 23473 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.168351 23703 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:48.181974 23715 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.182749 23618 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.186546 23682 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.151161 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:48.312602 23703 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:48.322407 23715 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:48.192337 23663 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.000s
I20260709 12:18:48.323506 23618 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:48.200054 18448 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:18:48.324059 23663 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:48.324380 23604 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.324160 23618 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.324591 23663 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.324919 23604 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.326843 23663 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.327615 18372 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:18:48.327845 23610 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.194612 23478 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:18:48.209187 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.328428 23478 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:48.328900 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.290719 19038 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:48.156252 23693 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.329828 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.330602 18448 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:48.331349 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.331468 23693 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.332152 18448 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:48.335278 23663 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:48.335678 23663 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:48.336030 23663 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.201951 23756 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 12:18:48.338480 23737 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.207383 23550 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.304787 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.309445 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.198971 23587 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.288846 23627 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.299790 23472 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:48.322363 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 171242us
I20260709 12:18:48.358330 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:48.323385 23715 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.327986 18372 pending_rounds.cc:85] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 9
I20260709 12:18:48.250159 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.360709 18372 pending_rounds.cc:107] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:48.361153 18372 raft_consensus.cc:2889] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.206705 23716 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.194831 23681 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.291833 23659 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:48.322875 23503 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.734s	user 0.005s	sys 0.005s
I20260709 12:18:48.350371 18449 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:48.350371 18443 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:48.355692 19041 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:48.396736 18291 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:48.251996 23539 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.273669 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.341986 19043 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:48.306533 23753 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:48.408222 18296 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:48.358125 23651 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:48.342581 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 180958us
I20260709 12:18:48.413210 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:48.313983 23722 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:48.395517 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 185655us
W20260709 12:18:48.399775 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 190079us
I20260709 12:18:48.400152 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.357180 23550 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:48.360862 23747 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
I20260709 12:18:48.339689 23542 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.328864 23478 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:48.362593 23715 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.306672 23741 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 12:18:48.424316 18444 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:48.458683 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.432435 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.362850 23627 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.446381 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.451982 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:48.441157 18440 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:48.365795 23749 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:48.351538 23703 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.605257 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.459487 19043 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:48.609308 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.460166 19041 raft_consensus.cc:2368] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11.
W20260709 12:18:48.616999 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 203846us
I20260709 12:18:48.459493 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:48.615432 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1137) took 1171 ms. Trace:
I20260709 12:18:48.618429 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.610491 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:48.619421 18367 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:48.617045 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.621150 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.618712 19040 rpcz_store.cc:276] 0709 12:18:47.444160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.444528 (+   368us) service_pool.cc:224] Handling call
0709 12:18:47.476367 (+ 31839us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:47.558778 (+ 82411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:47.558819 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:47.579736 (+ 20917us) log.cc:844] Serialized 78 byte log entry
0709 12:18:47.708250 (+128514us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:47.708290 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.708370 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:48.615054 (+906684us) raft_consensus.cc:1685] finished
0709 12:18:48.615086 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.615338 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46369,"raft.queue_time_us":46758,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":35048,"thread_start_us":27757,"threads_started":1,"wal-append.queue_time_us":279204}
I20260709 12:18:48.622547 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.624675 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.459973 23766 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.629729 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:48.599386 23762 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:18:48.634945 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.636646 18412 leader_election.cc:400] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11.
W20260709 12:18:48.599941 23764 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:18:48.647603 23427 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.648805 23472 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.651345 23478 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:48.652570 23668 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.000s
I20260709 12:18:48.652985 23668 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.653643 23478 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.654313 19037 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.632982 23760 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:18:48.635677 23530 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:48.656805 23530 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.657143 23530 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:48.657267 23715 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.658725 23559 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.639459 18375 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:18:48.659598 23519 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.632632 23542 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.660451 23542 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.662523 23542 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:48.640522 23763 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:48.642673 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2556) took 1205 ms. Trace:
I20260709 12:18:48.600930 23659 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.663769 18300 rpcz_store.cc:276] 0709 12:18:47.437666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.459204 (+ 21538us) service_pool.cc:224] Handling call
0709 12:18:48.642510 (+1183306us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54001cb6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:48.642616 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510188800}
I20260709 12:18:48.646857 23732 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.665196 18300 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:48.665845 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 989) took 1205 ms. Trace:
I20260709 12:18:48.665990 23732 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.666199 18300 rpcz_store.cc:276] 0709 12:18:47.460592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.664691 (+1204099us) service_pool.cc:224] Handling call
0709 12:18:48.664967 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.665600 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.665794 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.666671 23732 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:48.637048 23680 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.667276 23732 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.667704 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2541) took 1343 ms. Trace:
I20260709 12:18:48.667520 23680 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.668080 18298 rpcz_store.cc:276] 0709 12:18:47.324400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.324527 (+   127us) service_pool.cc:224] Handling call
0709 12:18:48.667555 (+1343028us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54002747d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:48.667651 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132490496}
I20260709 12:18:48.669282 18298 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:48.643548 23682 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:48.669790 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 991) took 1208 ms. Trace:
I20260709 12:18:48.670260 23682 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:48.671924 23655 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.000s
I20260709 12:18:48.671589 23682 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.644001 23681 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.648864 18412 leader_election.cc:403] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:48.670238 18298 rpcz_store.cc:276] 0709 12:18:47.461006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.668859 (+1207853us) service_pool.cc:224] Handling call
0709 12:18:48.669136 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.669599 (+   463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.669744 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.453534 23651 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:48.547413 23770 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:48.643918 23473 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:48.679554 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 990) took 1218 ms. Trace:
I20260709 12:18:48.696125 18300 rpcz_store.cc:276] 0709 12:18:47.460799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.678643 (+1217844us) service_pool.cc:224] Handling call
0709 12:18:48.678960 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.679157 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.679187 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.679245 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:48.679298 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.679366 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.679481 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.655066 23668 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.688690 23721 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.689488 23715 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.689947 18293 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:48.691943 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.705804 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.675959 18448 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.694164 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 994) took 1232 ms. Trace:
I20260709 12:18:48.694815 18371 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:48.706378 18367 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:48.419595 23697 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.639566 23587 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:48.699417 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:48.700569 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 64039.7us
I20260709 12:18:48.709579 18370 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:48.425282 23618 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.667073 23580 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.669711 23680 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.415942 23753 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.691210 23622 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:48.706518 18298 rpcz_store.cc:276] 0709 12:18:47.461733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.692063 (+1230330us) service_pool.cc:224] Handling call
0709 12:18:48.692432 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.692649 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.692686 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.692744 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:48.692800 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.692871 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.694113 (+  1242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.690946 23750 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.706899 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.693269 23651 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.732070 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.732693 18375 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:48.734302 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:48.739249 19038 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:18:48.739682 19038 pending_rounds.cc:85] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 11
I20260709 12:18:48.739907 19038 pending_rounds.cc:107] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:18:48.740255 19038 raft_consensus.cc:2889] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.739809 23732 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.741534 23550 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.746078 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:48.746680 18367 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.748436 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.705231 23681 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.765087 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.767043 19038 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:48.768393 18298 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.769068 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 996) took 1306 ms. Trace:
I20260709 12:18:48.769368 18298 rpcz_store.cc:276] 0709 12:18:47.462140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.729745 (+1267605us) service_pool.cc:224] Handling call
0709 12:18:48.768225 (+ 38480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.768806 (+   581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.769015 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.696007 23663 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.782487 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.795451 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.796427 18367 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:48.744233 23618 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:48.744822 23651 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:48.798959 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50214.6us
I20260709 12:18:48.798839 18298 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:48.801389 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1004) took 1321 ms. Trace:
I20260709 12:18:48.802215 18298 rpcz_store.cc:276] 0709 12:18:47.480471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.770137 (+1289666us) service_pool.cc:224] Handling call
0709 12:18:48.798671 (+ 28534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.801066 (+  2395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.801313 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.803678 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:48.780509 23734 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.000s
I20260709 12:18:48.819209 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1005) took 1338 ms. Trace:
I20260709 12:18:48.833696 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.805223 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.833617 18298 rpcz_store.cc:276] 0709 12:18:47.480790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.803272 (+1322482us) service_pool.cc:224] Handling call
0709 12:18:48.818505 (+ 15233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.818759 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.818797 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.818861 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:18:48.818908 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.818979 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.819130 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.807722 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:48.834363 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 126855us
I20260709 12:18:48.806241 23587 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.838524 18449 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:48.703295 23770 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.839658 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2160) took 1358 ms. Trace:
I20260709 12:18:48.749228 23750 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.750099 23622 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.847870 18298 rpcz_store.cc:276] 0709 12:18:47.481497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.838217 (+1356720us) service_pool.cc:224] Handling call
0709 12:18:48.839037 (+   820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.839211 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.839270 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.839315 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:48.839346 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.839410 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.839539 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.850600 23587 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.806653 23759 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.851450 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.840008 23716 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.862017 18370 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:48.863327 18298 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:48.889536 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2164) took 1406 ms. Trace:
I20260709 12:18:48.889847 18298 rpcz_store.cc:276] 0709 12:18:47.482755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.852153 (+1369398us) service_pool.cc:224] Handling call
0709 12:18:48.862901 (+ 10748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.889227 (+ 26326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.889486 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.856769 23587 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.864866 23663 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.834632 23734 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.866634 23750 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:48.867038 18291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2549) took 1526 ms (client timeout 1893 ms). Trace:
I20260709 12:18:48.867947 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2424) took 1986 ms. Trace:
I20260709 12:18:48.868233 19039 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:48.935175 19039 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:48.936262 18298 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:48.800107 23473 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:18:48.890688 23693 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:48.874313 18369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1165) took 2417 ms (client timeout 2902 ms). Trace:
I20260709 12:18:48.875880 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2165) took 2696 ms. Trace:
I20260709 12:18:48.878798 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 983) took 1419 ms. Trace:
I20260709 12:18:48.879487 18297 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:48.941254 18295 rpcz_store.cc:276] 0709 12:18:47.459150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.195370 (+736220us) service_pool.cc:224] Handling call
0709 12:18:48.195727 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:48.196130 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.196160 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:48.204264 (+  8104us) log.cc:844] Serialized 52 byte log entry
0709 12:18:48.393300 (+189036us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:48.410468 (+ 17168us) log.cc:844] Serialized 14 byte log entry
0709 12:18:48.878123 (+467655us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b1800240880. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:18:48.878327 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.878411 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:48.878460 (+    49us) raft_consensus.cc:1685] finished
0709 12:18:48.878576 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:48.878736 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46918,"prepare.queue_time_us":17247,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5453,"raft.queue_time_us":475777,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":7033,"spinlock_wait_cycles":7505792,"thread_start_us":83488,"threads_started":1,"wal-append.queue_time_us":282060}
I20260709 12:18:48.880570 18449 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:48.882730 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1136) took 1448 ms. Trace:
I20260709 12:18:48.843966 23550 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.884546 23684 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.884965 23770 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:48.886834 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.888398 18300 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.949236 18440 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:48.950330 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:48.875123 18294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2539) took 1555 ms (client timeout 1842 ms). Trace:
I20260709 12:18:48.848640 23694 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:18:48.916342 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 69355.8us
I20260709 12:18:48.806124 23618 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:48.830557 23530 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.934572 18366 rpcz_store.cc:276] 0709 12:18:46.882231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.882351 (+   120us) service_pool.cc:224] Handling call
0709 12:18:46.882652 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.603951 (+721299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.867633 (+1263682us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54002f8854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:48.867864 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464739712}
W20260709 12:18:48.923503 23679 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.867s	user 0.012s	sys 0.000s
I20260709 12:18:48.938047 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:48.871380 23744 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:48.934454 18291 rpcz_store.cc:269] 0709 12:18:47.340242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.340367 (+   125us) service_pool.cc:224] Handling call
0709 12:18:48.866832 (+1526465us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400090454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:48.866974 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427173120}
I20260709 12:18:48.938935 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2168) took 1455 ms. Trace:
I20260709 12:18:48.936019 19039 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:48.941130 18372 rpcz_store.cc:276] 0709 12:18:46.179830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.179993 (+   163us) service_pool.cc:224] Handling call
0709 12:18:46.180363 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.361792 (+2181429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.875625 (+513833us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b5400395654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:48.875798 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4937351552}
I20260709 12:18:48.943912 19036 rpcz_store.cc:276] 0709 12:18:47.434911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.435142 (+   231us) service_pool.cc:224] Handling call
0709 12:18:47.441978 (+  6836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.272503 (+830525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.882520 (+610017us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b5400603454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:48.882675 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564997504}
W20260709 12:18:48.946233 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.940953 18369 rpcz_store.cc:269] 0709 12:18:46.456930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.458797 (+  1867us) service_pool.cc:224] Handling call
0709 12:18:48.874106 (+2415309us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:48.874228 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114240768}
I20260709 12:18:48.922191 23622 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:48.947722 18295 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:18:48.946444 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 255562us
W20260709 12:18:48.951406 18294 rpcz_store.cc:269] 0709 12:18:47.319983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.320176 (+   193us) service_pool.cc:224] Handling call
0709 12:18:48.874914 (+1554738us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54000398d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:48.875063 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541766016}
I20260709 12:18:49.011178 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2169) took 1526 ms. Trace:
I20260709 12:18:48.980017 18371 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:18:49.006819 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.007279 18298 rpcz_store.cc:276] 0709 12:18:47.484003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.890618 (+1406615us) service_pool.cc:224] Handling call
0709 12:18:48.935962 (+ 45344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.936616 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.938878 (+  2262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.953253 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:48.940155 23693 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.952656 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:48.915126 23759 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.021095 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:49.021891 18440 raft_consensus.cc:2376] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16. Re-sending same reply.
I20260709 12:18:49.011552 18295 rpcz_store.cc:276] 0709 12:18:47.484281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.944865 (+1460584us) service_pool.cc:224] Handling call
0709 12:18:48.945136 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.010843 (+ 65707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.011125 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:49.008013 18375 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:49.013995 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.026041 19040 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.030061 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:48.946818 23655 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:48.983939 23779 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:18:49.045826 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2171) took 1519 ms. Trace:
I20260709 12:18:49.050637 18294 rpcz_store.cc:276] 0709 12:18:47.526787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.011434 (+1484647us) service_pool.cc:224] Handling call
0709 12:18:49.045148 (+ 33714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.045358 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.045407 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.045460 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:49.045503 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.045633 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.045759 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:49.041628 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 88454.3us
I20260709 12:18:49.036734 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.054199 18375 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:49.055807 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.056675 18372 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:49.061228 18298 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:49.061905 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2559) took 1418 ms. Trace:
I20260709 12:18:49.062054 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.062233 18298 rpcz_store.cc:276] 0709 12:18:47.643169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.012274 (+1369105us) service_pool.cc:224] Handling call
0709 12:18:49.059528 (+ 47254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.061686 (+  2158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.061854 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:48.952946 23694 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.067188 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.030014 23693 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.096185 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:49.098187 23790 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:49.102272 19040 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:49.066941 23761 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:49.105255 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.072027 23694 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:49.021174 23679 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.105921 19035 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:49.105868 18371 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:49.106614 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.117728 18372 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:49.086210 23682 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:49.111729 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.121528 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2170) took 1637 ms. Trace:
I20260709 12:18:49.121767 18369 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.112412 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.019841 23744 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.042093 23721 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:18:49.087710 23753 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.158205 18445 raft_consensus.cc:2368] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14.
I20260709 12:18:49.119412 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:49.122037 18291 rpcz_store.cc:276] 0709 12:18:47.501122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.008303 (+1507181us) service_pool.cc:224] Handling call
0709 12:18:49.008687 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.008955 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:49.019627 (+ 10672us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.120711 (+101084us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.120778 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:49.120845 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.120953 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.121463 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20810,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":100629,"threads_started":1}
I20260709 12:18:49.060025 23655 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.134971 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:49.165503 23792 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:18:49.167474 23788 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:18:49.169701 23679 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.120265 23550 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:49.134392 23682 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.171712 23550 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.173149 23734 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:49.174273 23530 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.176433 23721 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:49.176956 23721 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:49.164017 23539 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:49.180287 23539 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:49.180816 23539 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:49.181993 23539 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.183600 23618 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:49.185580 23539 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.185811 23520 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.186918 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1233) took 1817 ms. Trace:
I20260709 12:18:49.187317 18441 rpcz_store.cc:276] 0709 12:18:47.369042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.411110 (+ 42068us) service_pool.cc:224] Handling call
0709 12:18:49.186745 (+1775635us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:49.186852 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766662912}
W20260709 12:18:49.189397 23771 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.611s	user 0.000s	sys 0.001s
I20260709 12:18:49.198247 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.200860 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.201686 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.205199 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.208243 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:49.214411 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.178108 23721 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.222285 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.223776 19042 raft_consensus.cc:2410] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:18:49.156643 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59354.1us
I20260709 12:18:49.222149 23790 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:49.232594 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.233197 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:49.242799 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.252660 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:49.270063 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 150709us
I20260709 12:18:49.254138 19035 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
W20260709 12:18:49.233925 23781 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.286s	user 0.003s	sys 0.000s
I20260709 12:18:49.309724 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.315672 19040 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:49.319758 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.323400 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.176446 23552 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:49.317735 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:49.144739 23473 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 12:18:49.347815 19035 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.349215 23529 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:49.349685 23529 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.350049 23529 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.350204 23703 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.305301 23542 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:49.305796 23734 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:18:49.345757 23136 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.388568 23136 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.389918 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2693) took 1384 ms. Trace:
I20260709 12:18:49.390652 23136 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:49.392655 18293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 997) took 1930 ms (client timeout 2222 ms). Trace:
W20260709 12:18:49.393183 18293 rpcz_store.cc:269] 0709 12:18:47.462349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.973333 (+510984us) service_pool.cc:224] Handling call
0709 12:18:49.392377 (+1419044us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54001cc5d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:49.392584 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527516544}
I20260709 12:18:49.317051 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:49.396381 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 236014us
I20260709 12:18:49.400621 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2432) took 2511 ms. Trace:
I20260709 12:18:49.390322 18447 rpcz_store.cc:276] 0709 12:18:48.005096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.007455 (+  2359us) service_pool.cc:224] Handling call
0709 12:18:48.007774 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.008012 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:48.110464 (+102452us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.389400 (+1278936us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b18004d3bc0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a
0709 12:18:49.389587 (+   187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.389631 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:49.389678 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.389739 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.389853 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138924,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":5313,"raft.queue_time_us":82,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":79591168,"thread_start_us":161770,"threads_started":2,"wal-append.queue_time_us":282829}
I20260709 12:18:49.396184 18369 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:49.371001 23721 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.401069 18368 rpcz_store.cc:276] 0709 12:18:46.889111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.243941 (+354830us) service_pool.cc:224] Handling call
0709 12:18:47.315893 (+ 71952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.864380 (+548487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.400410 (+1536030us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540034a654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:49.400563 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921941760}
I20260709 12:18:49.395287 23655 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:49.440455 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.395394 23542 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:49.439778 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:49.459489 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 217142us
I20260709 12:18:49.460381 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:49.461189 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 137893us
I20260709 12:18:49.461318 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.462459 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:49.463845 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.465092 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.465706 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.467900 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.468254 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.469202 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.473651 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:49.469538 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.480065 18368 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:18:49.481225 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:49.485380 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.485755 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.492654 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.496861 19040 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:49.500545 22945 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:18:49.499075 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47285.2us
I20260709 12:18:49.503548 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:49.507148 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.508229 18375 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:49.522841 23675 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:18:49.523638 23734 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.480389 23761 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:49.524120 23734 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:49.524597 23734 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.524832 23761 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.455507 23622 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:49.527333 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.506722 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.530931 22945 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:49.528193 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.531668 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.529269 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:49.533896 18370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.025s	sys 0.034s
W20260709 12:18:49.534409 18370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.025s	sys 0.034s
I20260709 12:18:49.482434 23520 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:49.535760 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:49.491953 23737 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:18:49.542984 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.542053 22945 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.546640 23737 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.547147 23472 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:49.547503 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.548657 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.499887 23629 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:49.551622 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.552716 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:49.459458 23679 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:18:49.554512 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.523921 23704 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:49.554672 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:49.493584 23473 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:18:49.547546 23472 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:49.583458 23797 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:18:49.461580 23790 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.585695 19044 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:49.582603 19035 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:49.588166 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:49.588610 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116774us
I20260709 12:18:49.592250 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.566215 23520 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:49.574620 23759 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:49.613200 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.588114 23472 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:49.618315 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63732.5us
I20260709 12:18:49.619885 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:49.621665 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1228) took 2254 ms (client timeout 1645 ms). Trace:
W20260709 12:18:49.622072 18442 rpcz_store.cc:269] 0709 12:18:47.364136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.365839 (+  1703us) service_pool.cc:224] Handling call
0709 12:18:49.618316 (+2252477us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54002ab554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:49.618416 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700766464}
I20260709 12:18:49.592120 23682 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.591090 23550 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.605602 23601 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:18:49.612727 23136 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:49.658028 22945 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:18:49.632052 23651 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.025s
W20260709 12:18:49.669802 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 134110us
I20260709 12:18:49.615181 23618 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 12:18:49.673877 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 170648us
I20260709 12:18:49.675662 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:49.681753 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:49.682811 23473 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:49.683048 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.683348 23473 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.683836 23473 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:49.684000 23550 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:49.684417 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:49.685168 23550 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.685527 23550 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.685842 23550 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.685827 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.686829 23682 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:49.688223 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:49.689671 23682 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:49.691052 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:49.693073 23682 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:49.693647 23682 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:49.694615 23682 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.588121 23542 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.674131 23136 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.658779 23525 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:18:49.708335 23794 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:18:49.719673 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.709513 23136 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.707551 23675 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:49.709105 23662 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:49.697269 23651 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.740036 23741 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:49.747634 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 995) took 2285 ms. Trace:
I20260709 12:18:49.748255 18300 rpcz_store.cc:276] 0709 12:18:47.461934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.697668 (+1235734us) service_pool.cc:224] Handling call
0709 12:18:48.888165 (+190497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.949149 (+ 60984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.747341 (+798192us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400090450. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:49.747546 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437276416}
W20260709 12:18:49.751688 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87894.1us
I20260709 12:18:49.752041 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:49.752583 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.754702 18366 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.755290 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:49.757500 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 919) took 1006 ms. Trace:
I20260709 12:18:49.743366 23525 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:49.761575 18369 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 13. Ops: []
I20260709 12:18:49.757862 19043 rpcz_store.cc:276] 0709 12:18:48.751055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.751396 (+   341us) service_pool.cc:224] Handling call
0709 12:18:48.762905 (+ 11509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.938762 (+175857us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:48.938791 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.939222 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:18:49.223400 (+284178us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:49.223433 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.223503 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.757303 (+533800us) raft_consensus.cc:1685] finished
0709 12:18:49.757320 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.757434 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7632,"raft.queue_time_us":38742,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":27631,"thread_start_us":456487,"threads_started":2,"wal-append.queue_time_us":432343}
I20260709 12:18:49.704020 23520 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:49.743371 23811 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.763413 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:49.771538 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 987) took 2311 ms. Trace:
I20260709 12:18:49.751757 23820 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.775380 18375 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:49.773169 18292 rpcz_store.cc:276] 0709 12:18:47.460138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.663887 (+1203749us) service_pool.cc:224] Handling call
0709 12:18:48.664241 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.937914 (+273673us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:48.948353 (+ 10439us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.133923 (+185570us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.134317 (+   394us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.145074 (+ 10757us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:49.160979 (+ 15905us) log.cc:844] Serialized 26 byte log entry
0709 12:18:49.161154 (+   175us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:49.161275 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.161354 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.771266 (+609912us) raft_consensus.cc:1685] finished
0709 12:18:49.771300 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.771458 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":338623,"prepare.queue_time_us":12009,"raft.queue_time_us":311902,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":34136,"spinlock_wait_cycles":35849984,"thread_start_us":119090,"threads_started":1,"wal-append.queue_time_us":365975}
W20260709 12:18:49.780678 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.784709 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 109165us
W20260709 12:18:49.775372 23806 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:18:49.788064 18338 leader_election.cc:336] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.553s (SENT)
I20260709 12:18:49.762833 23525 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:49.792832 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:49.774796 23811 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:49.802588 23811 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.774513 23703 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:49.781922 23818 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:18:49.751159 23741 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.784226 23820 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:49.837366 23813 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.098s	user 0.001s	sys 0.003s
W20260709 12:18:49.837108 23804 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:18:49.843485 23550 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:49.848661 23651 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:49.849413 23817 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:49.850509 23604 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.851042 23734 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.851605 23820 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.848412 23525 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.861713 23741 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:49.864626 23587 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.932s	user 0.002s	sys 0.033s
W20260709 12:18:49.865279 23693 consensus_meta.cc:287] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.764s	user 0.026s	sys 0.009s
I20260709 12:18:49.865706 23693 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:49.867300 23681 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.094s	user 0.018s	sys 0.016s
I20260709 12:18:49.867535 23737 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:49.867726 23693 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:49.865067 23587 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:49.868883 23539 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.683s	user 0.016s	sys 0.015s
W20260709 12:18:49.869325 23694 consensus_meta.cc:287] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.021s
I20260709 12:18:49.869876 23694 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.870337 23696 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:49.870265 23587 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:49.867736 23681 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11.
I20260709 12:18:49.874112 23734 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:49.875666 23805 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.475s	user 0.000s	sys 0.004s
I20260709 12:18:49.871078 23803 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:49.877653 23803 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:49.884562 23823 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:49.885142 23823 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:49.885210 23803 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:49.886322 23803 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.886247 23823 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:49.888602 23787 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.013s	user 0.002s	sys 0.000s
W20260709 12:18:49.892344 23780 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.018s	user 0.000s	sys 0.005s
I20260709 12:18:49.894018 23823 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:49.925413 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:49.944139 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 178249us
W20260709 12:18:49.946902 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 158700us
W20260709 12:18:49.948216 18338 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.631s (SENT)
W20260709 12:18:49.966707 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.976429 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2189) took 1035 ms. Trace:
I20260709 12:18:49.976610 18368 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:49.977063 18370 rpcz_store.cc:276] 0709 12:18:48.940404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.940563 (+   159us) service_pool.cc:224] Handling call
0709 12:18:48.940849 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.941179 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.941205 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.942930 (+  1725us) log.cc:844] Serialized 26 byte log entry
0709 12:18:49.534863 (+591933us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:49.537442 (+  2579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.537530 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.975574 (+438044us) raft_consensus.cc:1685] finished
0709 12:18:49.975653 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.976121 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6693,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":3876,"thread_start_us":548561,"threads_started":1,"wal-append.queue_time_us":652761}
I20260709 12:18:49.850569 23472 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:49.863124 23550 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:50.013213 23601 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:50.013989 23472 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.015681 23675 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.006963 23473 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.030172 23694 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:50.032302 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.032415 23473 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:50.040052 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51200.1us
I20260709 12:18:50.044245 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.045434 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.045967 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.046455 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.046787 18298 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.047466 18300 raft_consensus.cc:2376] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11. Re-sending same reply.
I20260709 12:18:50.048341 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.048991 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.050340 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.052950 19036 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:18:50.052578 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.060766 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.062793 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.063654 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.064769 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2705) took 1634 ms. Trace:
I20260709 12:18:50.068639 18443 rpcz_store.cc:276] 0709 12:18:48.430188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.623952 (+193764us) service_pool.cc:224] Handling call
0709 12:18:48.632038 (+  8086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.699704 (+ 67666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.699735 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.702169 (+  2434us) log.cc:844] Serialized 26 byte log entry
0709 12:18:48.830580 (+128411us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:48.830636 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.832117 (+  1481us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.888081 (+1055964us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:50.051032 (+162951us) raft_consensus.cc:1685] finished
0709 12:18:50.064149 (+ 13117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.064533 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3382,"raft.queue_time_us":3475,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":30765824,"thread_start_us":79402,"threads_started":1,"wal-append.queue_time_us":145498}
I20260709 12:18:50.072871 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.062961 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.083943 23472 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.122463 23472 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.125644 23472 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.101466 23675 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.102101 18366 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:50.126868 23820 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:18:50.107566 23737 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.129035 23737 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:50.131448 18298 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:50.132221 23811 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.132660 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.143360 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2553) took 2767 ms. Trace:
I20260709 12:18:50.143786 18296 rpcz_store.cc:276] 0709 12:18:47.376263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.376461 (+   198us) service_pool.cc:224] Handling call
0709 12:18:47.376715 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.408829 (+1032114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.142900 (+1734071us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002364d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.143148 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2366916480}
I20260709 12:18:50.144327 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.101341 23703 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:50.145676 23703 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.148030 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 986) took 2688 ms. Trace:
I20260709 12:18:50.148880 23703 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.150192 18298 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.151424 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.152864 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.153403 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:50.141976 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2177) took 2498 ms. Trace:
I20260709 12:18:50.156333 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.148478 18297 rpcz_store.cc:276] 0709 12:18:47.459921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.196894 (+736973us) service_pool.cc:224] Handling call
0709 12:18:48.879323 (+682429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.942815 (+ 63492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.147580 (+1204765us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54000398d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:50.147838 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568675200}
I20260709 12:18:50.145164 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.162707 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.160756 18300 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
I20260709 12:18:50.168179 18293 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.169718 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.170948 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.174589 18447 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:50.154013 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:50.175737 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:50.101336 23684 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.118s	user 0.017s	sys 0.023s
I20260709 12:18:50.208791 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.209690 18444 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.190214 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:50.211082 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.255512 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:50.256321 18370 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.260156 23823 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.260581 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2706) took 1468 ms. Trace:
I20260709 12:18:50.261112 23811 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.260821 23823 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.210002 23559 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.210260 19006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.004s	sys 0.012s
I20260709 12:18:50.260994 18446 rpcz_store.cc:276] 0709 12:18:48.791765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.791917 (+   152us) service_pool.cc:224] Handling call
0709 12:18:48.831602 (+ 39685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.887482 (+ 55880us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:48.932103 (+ 44621us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.087166 (+155063us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.088012 (+   846us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.092297 (+  4285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.092340 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:49.092389 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.866792 (+774403us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b180041f260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:50.260161 (+393369us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b5400422f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.260221 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.260401 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7137,"prepare.queue_time_us":828730,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":13316,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":6474,"spinlock_wait_cycles":138179200,"thread_start_us":151223,"threads_started":1,"wal-append.queue_time_us":229649,"wal-append.run_cpu_time_us":2006,"wal-append.run_wall_time_us":1004395}
W20260709 12:18:50.269237 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51773.6us
I20260709 12:18:50.271466 18291 raft_consensus.cc:2393] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:50.211427 23529 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.023s
W20260709 12:18:50.272554 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.273536 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2406) took 2071 ms. Trace:
I20260709 12:18:50.275728 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:50.272001 18442 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:50.279753 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.280557 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:50.280742 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:50.289577 19006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.024s
I20260709 12:18:50.191138 23737 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:50.294240 18370 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:50.294831 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.297683 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.295516 18291 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:50.287971 18448 rpcz_store.cc:276] 0709 12:18:48.202090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.339770 (+137680us) service_pool.cc:224] Handling call
0709 12:18:48.343299 (+  3529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.711246 (+367947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.272929 (+1561683us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54000d8c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.273332 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764534528}
I20260709 12:18:50.294279 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:50.297312 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 626172us
I20260709 12:18:50.306044 18297 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.305994 18374 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.155966 18295 rpcz_store.cc:276] 0709 12:18:47.643733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.023699 (+1379966us) service_pool.cc:224] Handling call
0709 12:18:49.024041 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.024287 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:49.306487 (+282200us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.141157 (+834670us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b180038a580. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287
0709 12:18:50.141343 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.141416 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:50.141548 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.141631 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.141776 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14762,"prepare.queue_time_us":39744,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":88443,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":12776,"spinlock_wait_cycles":40060800,"thread_start_us":627542,"threads_started":3,"wal-append.queue_time_us":311973}
I20260709 12:18:50.283788 23539 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Committing config change with OpId 10.6: config changed from index -1 to 6, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:50.305657 18366 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.310524 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:50.311965 23841 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:18:50.316082 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.317099 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.319521 18291 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.320186 18449 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:18:50.321139 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.321429 23629 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:50.321612 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.322125 23629 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.322510 18447 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:50.322731 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.325644 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.327697 18442 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:50.328941 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48589.6us
I20260709 12:18:50.329866 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.331221 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.331597 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 916) took 1621 ms. Trace:
I20260709 12:18:50.331919 18297 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:50.333003 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.332000 19037 rpcz_store.cc:276] 0709 12:18:48.710490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.710636 (+   146us) service_pool.cc:224] Handling call
0709 12:18:48.743537 (+ 32901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.886358 (+142821us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:49.105833 (+219475us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.482748 (+376915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.482834 (+    86us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:49.482901 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.294879 (+811978us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400546854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.299103 (+  4224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.331405 (+ 32302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15040,"prepare.queue_time_us":34639,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":3354,"raft.queue_time_us":27117,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":24157,"spinlock_wait_cycles":327860096,"thread_start_us":585345,"threads_started":2,"wal-append.queue_time_us":459712}
I20260709 12:18:50.326705 18298 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.334197 18370 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:50.302271 23559 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.335180 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.323567 18295 raft_consensus.cc:2410] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:50.300282 23529 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.338402 18299 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:50.339814 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.340641 18298 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 10.
W20260709 12:18:50.341795 19006 leader_election.cc:336] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.671s (SENT)
I20260709 12:18:50.321362 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.363539 23675 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.363415 18440 raft_consensus.cc:2368] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15.
I20260709 12:18:50.341914 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.365141 18298 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:50.366080 23823 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.366251 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.369788 18291 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:50.370770 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.371218 23840 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.371928 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.374774 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.375783 18291 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:50.377130 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.377465 23786 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:50.375489 23587 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:50.379671 23829 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:50.379724 23587 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.380231 23829 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:50.381098 23587 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.381451 23829 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:50.385265 23136 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.007s
I20260709 12:18:50.385365 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.385823 23136 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:50.387331 18297 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.388618 23710 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:18:50.386744 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 90172.7us
I20260709 12:18:50.344986 18370 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:50.397740 23744 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.398408 18366 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:50.371963 23840 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.398420 23529 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.402356 18295 raft_consensus.cc:2376] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13. Re-sending same reply.
I20260709 12:18:50.403537 23840 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.383008 23829 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.408517 23136 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:50.413082 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.418498 23789 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.546s	user 0.000s	sys 0.005s
I20260709 12:18:50.422079 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.422777 18446 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.424609 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:50.425249 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.393157 18374 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:50.359854 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.432035 18298 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:50.433254 18443 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:50.435595 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:50.363539 23629 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.439584 18446 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:50.441144 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.442121 18298 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.430589 23840 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.468612 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 922) took 1667 ms. Trace:
I20260709 12:18:50.469080 19038 rpcz_store.cc:276] 0709 12:18:48.800798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.803098 (+  2300us) service_pool.cc:224] Handling call
0709 12:18:48.803539 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.803941 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.803982 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.804504 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:18:48.843105 (+ 38601us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:48.843162 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.843277 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.417388 (+1574111us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:50.467299 (+ 49911us) raft_consensus.cc:1685] finished
0709 12:18:50.467323 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.468414 (+  1091us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5822,"raft.queue_time_us":35145,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11012,"spinlock_wait_cycles":113171200,"thread_start_us":38182,"threads_started":1,"wal-append.queue_time_us":67106}
I20260709 12:18:50.472538 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:50.471918 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 126629us
I20260709 12:18:50.473160 18298 raft_consensus.cc:2376] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13. Re-sending same reply.
I20260709 12:18:50.456657 23529 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:18:50.478554 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51229.7us
I20260709 12:18:50.483312 23791 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.483963 23791 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.486276 23791 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.487280 23839 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:50.482709 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.472631 23136 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:50.494330 23675 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:50.498487 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1184) took 2800 ms. Trace:
I20260709 12:18:50.499788 23744 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:18:50.502671 18339 connection.cc:256] RPC call timeout handler was delayed by -1.5919218249998721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 12:18:50.510021 23721 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:50.510499 23721 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.511550 23721 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.512297 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.512799 23721 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:50.513000 18296 raft_consensus.cc:2376] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16. Re-sending same reply.
I20260709 12:18:50.517011 23627 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:50.519738 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.520273 23629 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.520982 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.521077 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.522151 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.524291 19038 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
W20260709 12:18:50.525342 23783 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.662s	user 0.000s	sys 0.005s
W20260709 12:18:50.526322 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.529381 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 927) took 1464 ms. Trace:
I20260709 12:18:50.530000 23675 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:50.530431 23675 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.531530 23675 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.500974 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.537034 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2186) took 1730 ms. Trace:
I20260709 12:18:50.538300 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.537436 18367 rpcz_store.cc:276] 0709 12:18:48.806556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.806810 (+   254us) service_pool.cc:224] Handling call
0709 12:18:48.807090 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.063724 (+256634us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:49.087141 (+ 23417us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.222348 (+135207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.222421 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:49.222467 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.536636 (+1314169us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400116054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.536692 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.536853 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2185,"prepare.queue_time_us":27743,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":11071,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":40605,"spinlock_wait_cycles":589755264,"thread_start_us":132582,"threads_started":1,"wal-append.queue_time_us":350624,"wal-append.run_cpu_time_us":1408,"wal-append.run_wall_time_us":1083633}
I20260709 12:18:50.539115 18293 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.539991 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.540639 18293 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:50.541393 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.541531 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.541992 18296 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:50.533761 23675 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.542826 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.529762 19036 rpcz_store.cc:276] 0709 12:18:49.064659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.067432 (+  2773us) service_pool.cc:224] Handling call
0709 12:18:49.090217 (+ 22785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.055427 (+965210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.529129 (+473702us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54004a56d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.529327 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065631488}
I20260709 12:18:50.545766 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.545866 23820 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:50.546844 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.547356 23601 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.547837 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.548012 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.547971 23601 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:50.549201 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.549762 18296 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:50.550482 23697 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:50.550750 23601 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.550920 18296 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:50.551605 23541 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:50.551762 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:50.549573 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.553995 23559 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.555605 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2563) took 2585 ms. Trace:
I20260709 12:18:50.557796 23722 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:50.546566 23820 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.558331 23821 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:50.557801 23815 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:50.498870 18373 rpcz_store.cc:276] 0709 12:18:47.694332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.696518 (+  2186us) service_pool.cc:224] Handling call
0709 12:18:47.696783 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.117825 (+421042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.117866 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:48.117917 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:48.117967 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.494654 (+2376687us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400114c54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.494728 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.494944 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967805824}
I20260709 12:18:50.560252 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:50.567718 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.542114 18293 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.552735 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.581741 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:50.577582 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.579053 23722 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.585611 23821 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:50.586066 23821 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.586138 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:50.587412 18291 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:50.582327 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:50.588646 23601 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:50.580191 23542 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:50.577265 18294 rpcz_store.cc:276] 0709 12:18:47.970156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.051980 (+1081824us) service_pool.cc:224] Handling call
0709 12:18:49.071048 (+ 19068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.089051 (+ 18003us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:49.113088 (+ 24037us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.875159 (+762071us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:50.553905 (+678746us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b1800377200. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c23a
0709 12:18:50.554480 (+   575us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.554854 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.555065 (+   211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.555147 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:50.555217 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.555331 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.555432 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22878,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"raft.queue_time_us":3252,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":242,"spinlock_wait_cycles":70686976,"thread_start_us":31010,"threads_started":1,"wal-append.queue_time_us":211455}
I20260709 12:18:50.591758 18296 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:50.595865 18297 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:50.598376 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:50.600852 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.538s (SENT)
W20260709 12:18:50.613209 18445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2694) took 2607 ms (client timeout 1887 ms). Trace:
I20260709 12:18:50.617064 19039 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.622308 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:50.623811 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:50.624696 19038 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:50.625595 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:50.626304 19037 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:50.628695 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:50.629247 19040 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.593292 23541 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:50.629906 23541 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.630903 23541 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.635012 23759 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:50.635856 23759 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.636047 19009 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:50.637034 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:50.636955 23759 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.644502 23542 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.625631 23136 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:50.646303 23136 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:50.645999 23542 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.634872 23703 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:50.613644 18445 rpcz_store.cc:269] 0709 12:18:48.005418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.008335 (+  2917us) service_pool.cc:224] Handling call
0709 12:18:50.612809 (+2604474us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540035fa54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:50.613024 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2504537984}
I20260709 12:18:50.588994 23539 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:50.597107 19007 leader_election.cc:400] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15.
I20260709 12:18:50.675801 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.676626 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.681059 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.682749 19007 leader_election.cc:403] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:50.683585 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:50.686254 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87941.4us
I20260709 12:18:50.690152 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.586189 23722 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:50.693322 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 140678us
I20260709 12:18:50.693322 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.694247 18445 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:50.694579 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.681245 23601 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.695351 18449 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:50.695890 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.698974 18444 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:50.699069 23587 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:50.699761 23587 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.700896 23587 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:50.707779 18441 raft_consensus.cc:2410] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:50.709607 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2444) took 1226 ms. Trace:
I20260709 12:18:50.710100 18369 rpcz_store.cc:276] 0709 12:18:49.483370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.483510 (+   140us) service_pool.cc:224] Handling call
0709 12:18:49.483778 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.763060 (+279282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.709121 (+946061us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54000f9e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.709407 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638450944}
I20260709 12:18:50.592741 23821 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.683224 23559 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:50.717244 18293 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:50.716162 18291 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:50.685990 23703 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.691020 23539 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.720728 19040 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:18:50.721078 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.721598 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.723932 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 144747us
I20260709 12:18:50.724536 23802 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:50.725559 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.729457 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 180036us
W20260709 12:18:50.734009 18299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2564) took 2721 ms (client timeout 2020 ms). Trace:
W20260709 12:18:50.734391 18299 rpcz_store.cc:269] 0709 12:18:48.012869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.063670 (+ 50801us) service_pool.cc:224] Handling call
0709 12:18:50.733808 (+2670138us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:50.733930 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5229090176}
I20260709 12:18:50.735605 23722 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:50.739972 23525 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.740754 23820 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.741705 19035 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:50.743055 23519 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.724792 23821 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.756814 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.730383 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:50.759601 18412 connection.cc:256] RPC call timeout handler was delayed by -1.1245507789999465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2838s for a response
I20260709 12:18:50.760991 23601 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.761633 23601 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.762771 19039 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
W20260709 12:18:50.763497 23714 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 12:18:50.763561 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.771790 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.779116 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.780568 18297 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
W20260709 12:18:50.784791 19039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 905) took 2704 ms (client timeout 1580 ms). Trace:
I20260709 12:18:50.786814 23803 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15.
I20260709 12:18:50.787122 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2203) took 1022 ms. Trace:
I20260709 12:18:50.787858 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:50.788596 18367 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:50.787634 18368 rpcz_store.cc:276] 0709 12:18:49.764899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.765053 (+   154us) service_pool.cc:224] Handling call
0709 12:18:49.783381 (+ 18328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.983564 (+200183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.775814 (+792250us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400168ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.787059 (+ 11245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441515904}
W20260709 12:18:50.790637 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60303.4us
W20260709 12:18:50.785343 19039 rpcz_store.cc:269] 0709 12:18:48.071681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.073057 (+  1376us) service_pool.cc:224] Handling call
0709 12:18:48.868112 (+795055us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b540066eca0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7fe18 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:50.616304 (+1748192us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b58001d1ed0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68dd80aa 00007f3c68e5a447 00007f3c68e72fe7 00007f3c68e747de 00007f3c68e8027c 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c
0709 12:18:50.775988 (+159684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2351235712}
W20260709 12:18:50.790748 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 96274.2us
I20260709 12:18:50.764227 23839 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.792184 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 204310us
I20260709 12:18:50.793236 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.793413 23539 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.795397 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:50.796240 23519 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:50.796965 23519 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:50.797245 23753 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.793236 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:50.799337 23839 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.792251 23820 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:50.800321 23753 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:50.800527 23820 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:50.800750 19036 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:50.800976 23820 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.804023 23721 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:50.804934 23703 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.805596 23721 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:50.806447 23703 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.806744 23721 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:50.808327 23721 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:50.808895 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 231777us
I20260709 12:18:50.822655 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.830021 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:50.832491 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:50.832721 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:50.833312 18367 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.835871 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:50.836771 23753 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:50.837268 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:50.837370 23753 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:50.838670 23753 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.839973 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:50.840728 23519 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:50.841068 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:50.841085 23519 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.841852 23519 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.841979 23815 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:50.842039 23697 raft_consensus.cc:2764] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:18:50.842617 23815 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:50.843039 23815 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:50.845901 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.847177 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.847750 18291 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:50.849073 23839 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.849717 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.850144 23601 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.850451 18294 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.847944 18299 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:50.852689 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.853417 18293 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:50.854535 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.858672 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:50.859624 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1148) took 1059 ms. Trace:
I20260709 12:18:50.860013 19043 rpcz_store.cc:276] 0709 12:18:49.800464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.800614 (+   150us) service_pool.cc:224] Handling call
0709 12:18:49.800922 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:49.802647 (+  1725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:49.802675 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:49.832356 (+ 29681us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.503301 (+670945us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18002b45c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:50.773549 (+270248us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:50.773615 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.791067 (+ 17452us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.798774 (+  7707us) raft_consensus.cc:1685] finished
0709 12:18:50.798793 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.859439 (+ 60646us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2940,"raft.queue_time_us":12652,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":2569,"spinlock_wait_cycles":4860800,"thread_start_us":184084,"threads_started":1,"wal-append.queue_time_us":925892}
I20260709 12:18:50.884462 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.847748 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:50.885747 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 125517us
W20260709 12:18:50.886168 18412 connection.cc:256] RPC call timeout handler was delayed by -1.0064177509998444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
W20260709 12:18:50.875468 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82361.5us
I20260709 12:18:50.883419 18367 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:50.859174 23815 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.892668 23815 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.894721 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.895462 18444 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:50.863942 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.867895 18371 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:18:50.868566 18373 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:50.904084 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.907577 18299 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 10: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:50.912256 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.913136 18449 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.913399 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.914088 18445 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 10: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:50.914557 18441 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.914776 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.914970 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.915629 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.920878 18449 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.902817 23541 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:50.921123 23848 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.135s	user 0.000s	sys 0.003s
I20260709 12:18:50.934837 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:50.940596 18299 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:50.935209 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:50.942055 18367 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:50.931874 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.936841 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:50.937635 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:50.924727 23850 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.256s	user 0.003s	sys 0.000s
I20260709 12:18:50.861382 23136 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:50.946543 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.949980 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:50.932549 23541 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.952836 18441 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:50.955348 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.956410 23682 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:50.956972 23682 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:50.957916 23682 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:50.958791 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:50.967760 23820 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:50.969960 23820 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:50.974995 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:50.976524 23136 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:50.977432 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:50.978055 18372 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:50.983003 18442 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:18:50.955644 23541 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.998618 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:50.998730 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:51.001271 18444 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:51.002429 18264 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:50.954888 23721 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:50.992430 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.009730 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.008138 23136 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:51.012527 23136 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:51.013484 23136 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.015350 18192 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.017267 18192 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:51.019651 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 82941.6us
I20260709 12:18:51.022995 18449 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:51.026885 23659 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:18:51.027667 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:51.031281 18292 raft_consensus.cc:2410] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 15 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:18:51.040179 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.041105 18291 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:51.062227 18295 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:51.026392 23820 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:51.039088 23587 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.088311 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:51.088990 23539 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:51.089560 23714 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.089807 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 81457us
I20260709 12:18:51.089927 23542 raft_consensus.cc:1064] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:18:51.084898 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.090758 23821 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:51.101084 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.101890 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:51.102334 23675 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.003s
I20260709 12:18:51.102897 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.103525 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.030761 23627 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:18:51.104143 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.105180 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.019037 18192 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 12, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 12 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.134162 23821 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.135586 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:51.140141 23853 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:18:51.139701 23542 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.140610 23821 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:51.141201 23816 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:51.141727 23816 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:51.142092 23816 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:51.151484 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.159192 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.160367 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.163952 18371 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:51.164659 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.166152 18368 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:51.157852 18367 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.168120 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.168816 18298 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:51.169539 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.170358 18370 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:51.171463 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.173352 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.176272 18375 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:51.177827 18369 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:51.181898 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.189397 18372 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:51.126607 23714 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.190421 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.197624 18367 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:51.200755 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 294087us
I20260709 12:18:51.195516 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.201886 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:51.203729 18294 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.186875 23822 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:51.214195 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.217929 18298 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:51.225189 18293 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10.
I20260709 12:18:51.188763 23587 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:51.228853 18296 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:51.231199 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.229225 18297 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:51.211733 23822 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.233770 18295 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:51.227314 23826 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:51.208020 23759 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:18:51.245743 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 357725us
W20260709 12:18:51.246532 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.250479 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.237725 23819 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:51.235312 23822 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:51.298774 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 381212us
W20260709 12:18:51.306226 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.305727 19006 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.543s (SENT)
I20260709 12:18:51.297762 23820 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:51.344636 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.305521 23587 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:51.349526 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 99166.7us
I20260709 12:18:51.294353 23519 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:51.349496 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:51.363900 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 249862us
I20260709 12:18:51.365460 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.306625 23816 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.351712 23525 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
W20260709 12:18:51.373968 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.347465 23659 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
W20260709 12:18:51.377624 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 76703.9us
W20260709 12:18:51.378340 19006 leader_election.cc:336] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.547s (SENT)
I20260709 12:18:51.378713 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.380632 23541 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.389640 18294 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:51.376963 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.381513 23627 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:51.407989 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:51.373699 23714 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:51.376737 23819 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:51.426863 18292 raft_consensus.cc:2368] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
W20260709 12:18:51.428165 23859 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.188s	user 0.005s	sys 0.000s
I20260709 12:18:51.428063 18192 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 6 to 9, term changed from 10 to 11, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.425032 23697 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:18:51.432338 23862 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:18:51.435453 23819 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:51.419582 23826 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:51.456887 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 265082us
I20260709 12:18:51.430423 18192 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 11, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 11 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:51.464603 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 94610.1us
I20260709 12:18:51.473594 18291 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
W20260709 12:18:51.509452 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.511327 23826 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.511994 23826 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.514597 23826 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:51.516040 23864 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:18:51.523358 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.442777 23819 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.530144 18299 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
W20260709 12:18:51.532048 23866 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:18:51.537338 23863 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.286s	user 0.003s	sys 0.000s
I20260709 12:18:51.536576 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.519795 23682 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:18:51.546046 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 223790us
W20260709 12:18:51.550379 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.561857 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 183042us
I20260709 12:18:51.546337 23682 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:51.570711 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:51.573019 18195 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 294) took 27547 ms (client timeout 15000 ms). Trace:
I20260709 12:18:51.595685 23872 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:51.596524 23872 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.599390 23541 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:51.547715 23870 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:18:51.556891 23601 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:51.561383 23821 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.556181 23519 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:51.622841 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.635876 18442 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:51.603892 23816 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:51.615329 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 210019us
I20260709 12:18:51.624786 18192 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 12, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.610548 23822 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.652666 18192 catalog_manager.cc:5697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 3 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.643894 23722 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:51.603163 23682 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:51.573546 18195 rpcz_store.cc:269] 0709 12:18:24.025829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.026076 (+   247us) service_pool.cc:224] Handling call
0709 12:18:51.572706 (+27546630us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:47.408758 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:47.588558 (+179800us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:47.593214 (+  4656us) write_op.cc:435] Acquired schema lock
0709 12:18:47.603762 (+ 10548us) tablet.cc:662] Decoding operations
0709 12:18:48.159691 (+555929us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:48.169313 (+  9622us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:48.171482 (+  2169us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:18:48.184460 (+ 12978us) tablet.cc:701] Row locks acquired
0709 12:18:48.184480 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:18:48.184627 (+   147us) write_op.cc:270] Start()
0709 12:18:48.184738 (+   111us) write_op.cc:276] Timestamp: P: 1783599528184581 usec, L: 0
0709 12:18:48.184759 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:18:48.185437 (+   678us) log.cc:844] Serialized 5536 byte log entry
0709 12:18:48.620989 (+435552us) op_driver.cc:464] REPLICATION: finished
0709 12:18:48.630475 (+  9486us) write_op.cc:301] APPLY: starting
0709 12:18:48.630592 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:49.500540 (+869948us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:49.527765 (+ 27225us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:49.813017 (+285252us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b1800266f80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:49.863346 (+ 50329us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:49.863645 (+   299us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:18:49.863688 (+    43us) write_op.cc:312] APPLY: finished
0709 12:18:49.864059 (+   371us) log.cc:844] Serialized 129 byte log entry
0709 12:18:49.894823 (+ 30764us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:49.894975 (+   152us) write_op.cc:454] Released schema lock
0709 12:18:49.895027 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:50.058277 (+163250us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:50.058421 (+   144us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":61132,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":9194,"apply.run_cpu_time_us":43051,"apply.run_wall_time_us":1427437,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62591,"num_ops":14,"prepare.queue_time_us":203353,"prepare.run_cpu_time_us":14037,"prepare.run_wall_time_us":823750,"raft.queue_time_us":282354,"raft.run_cpu_time_us":6251,"raft.run_wall_time_us":16627,"replication_time_us":436068,"spinlock_wait_cycles":3447168,"thread_start_us":377864,"threads_started":5,"wal-append.queue_time_us":151629,"wal-append.run_cpu_time_us":68837,"wal-append.run_wall_time_us":2272207}]]}
W20260709 12:18:51.660776 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.663353 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:51.681770 18442 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:51.664139 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:51.683084 18444 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:51.665354 18446 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:51.652206 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:51.671518 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 100933us
I20260709 12:18:51.674515 18448 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:18:51.686754 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.675719 23839 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:51.677083 23883 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:18:51.638960 23821 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.639350 23753 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:51.688297 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.696700 23721 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:18:51.704838 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:51.706490 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 138165us
I20260709 12:18:51.712198 23622 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:18:51.714232 23668 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:51.714608 23668 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:51.715474 23668 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.716638 23668 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:51.707988 23659 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:51.720306 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.721568 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.722398 23697 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:51.722787 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:51.722728 23880 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:51.726063 23880 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:51.705765 23682 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:51.739672 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.744395 23821 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:51.744817 18372 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:51.745900 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.747435 23622 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:51.748901 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.749742 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.751978 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:51.732137 23880 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.753104 23880 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:51.753665 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.725582 18192 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 13 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:18:51.733214 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45019.6us
I20260709 12:18:51.739387 23627 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:51.783361 23820 raft_consensus.cc:2764] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:18:51.784281 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:51.785660 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:51.787590 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.746047 23872 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:51.733035 23822 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:51.761945 18442 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:51.769485 23722 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:51.770864 23622 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:51.770542 18192 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.772249 23821 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:51.775368 23881 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:51.775738 23753 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:51.780085 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.781426 23721 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:51.782830 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 60128.5us
I20260709 12:18:51.792763 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:51.790807 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.802124 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.817258 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.806490 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.819005 18366 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.803541 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:51.836027 19043 raft_consensus.cc:606] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:51.801192 23659 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:51.829401 23541 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:51.838189 23659 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.841712 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:51.843969 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 64044.5us
I20260709 12:18:51.843780 18192 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.844779 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.845343 18192 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 12 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } }
I20260709 12:18:51.846267 23873 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:51.846735 23873 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:51.856348 18366 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:51.815867 23881 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.862264 23881 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:51.829335 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.797636 23627 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:51.852205 23821 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:51.859185 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:51.852049 23722 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:51.828274 23872 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:51.827482 23525 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:51.800097 23883 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.827586 23136 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:18:51.849633 23682 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:18:51.880017 23883 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.882655 23883 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:51.797161 23519 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:18:51.847163 23873 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:51.860739 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68086us
I20260709 12:18:51.877125 23697 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:51.847144 18192 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.905133 18192 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:51.857023 23659 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:51.830528 23839 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 12:18:51.887804 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 70709us
W20260709 12:18:51.885643 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59313.6us
I20260709 12:18:51.909406 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.909906 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:51.885912 23819 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:51.884652 23821 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:51.910487 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.911820 23894 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:51.911518 23821 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.912597 23894 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:51.914342 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:51.896375 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.901139 23722 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:51.918215 23722 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.918609 23519 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:51.902940 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.906800 18192 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 14, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 14 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:51.919912 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:51.903755 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44683.3us
I20260709 12:18:51.895298 23622 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:51.919044 23519 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.906620 23881 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.920162 23882 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.929416 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:51.930373 18294 raft_consensus.cc:2393] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:51.936439 18369 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:51.933468 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.943934 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.944644 18367 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:51.951786 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.954901 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:51.955699 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45863.7us
I20260709 12:18:51.922719 23622 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:51.957283 18370 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:51.959887 18370 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:51.954002 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:51.924233 23519 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.945151 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.925668 23753 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:18:51.958899 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 133481us
I20260709 12:18:51.966837 23722 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:51.968050 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:51.970816 23873 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:51.972198 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:51.974762 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.961472 23622 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:51.975661 18370 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:51.976764 18370 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.979344 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.980132 18375 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:51.981925 23819 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.982447 23819 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:51.983516 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:51.984066 19043 raft_consensus.cc:606] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.984728 23819 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:51.987402 23721 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:51.987759 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:51.988157 23873 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:51.988116 23721 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:51.989382 23822 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:51.989770 23822 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:51.990161 23822 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:51.995528 23839 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:51.953310 18192 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:51.997007 18192 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.998271 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:51.999142 23872 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:51.988986 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.005373 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:51.990867 23721 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:51.960764 23883 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:52.028189 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84223.7us
I20260709 12:18:52.029706 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:51.925048 23882 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.033576 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.035311 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.016574 18192 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 7 to 9, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 9 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:52.037947 18192 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 8, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 9 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:52.039851 18192 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 14, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:52.041767 18192 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 12 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:52.026500 18443 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:52.047613 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:52.053072 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53424.9us
I20260709 12:18:52.011049 23722 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.013827 23873 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:52.056758 23883 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.030843 23895 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:52.071532 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82614.7us
I20260709 12:18:52.072803 19006 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.073246 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:52.073849 19043 raft_consensus.cc:606] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.074033 23826 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.075047 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.076988 23882 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:52.077488 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.077845 23541 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.078609 23519 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.079939 23627 raft_consensus.cc:993] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:52.070118 23722 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.068420 23883 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.060230 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.044553 23896 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:52.090435 18370 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:52.067355 23873 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:52.108125 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.106909 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:52.109383 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76187.3us
I20260709 12:18:52.112540 19036 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:18:52.108991 19043 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:52.114847 18291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1f155085cb647ff957b468384227872"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:18:52.115489 18291 raft_consensus.cc:606] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.115813 18291 raft_consensus.cc:614] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:52.119371 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.121935 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:52.122538 19040 raft_consensus.cc:606] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.123783 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.125013 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.126170 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.073737 23822 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.130841 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:52.131364 19040 raft_consensus.cc:606] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.131774 19040 raft_consensus.cc:614] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:52.133059 18443 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:52.095014 23895 raft_consensus.cc:1081] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:52.108320 23873 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:52.162985 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.163190 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:52.166286 18370 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.130719 23753 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.168354 19036 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:52.169554 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 134351us
I20260709 12:18:52.165078 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:18:52.177532 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 121064us
I20260709 12:18:52.150262 23872 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:52.174602 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.182857 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.183065 23894 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:52.183333 23622 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:52.184310 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 58252us
I20260709 12:18:52.185182 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.185513 23622 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.185886 23884 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:52.186563 23884 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:52.187094 23884 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.187955 23896 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:52.189929 23894 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:52.190325 23894 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:52.191283 23894 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.192672 23894 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.197408 23839 raft_consensus.cc:993] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:52.197904 23839 raft_consensus.cc:1081] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:52.199299 23519 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.199503 23820 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.200233 23820 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.201844 18372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:18:52.202636 23820 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.202894 23872 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:52.203603 23872 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:52.217221 23890 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:52.217981 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.175859 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:52.226052 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:52.228297 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:52.226783 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.171471 23682 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.233258 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:52.245198 19036 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 9 LEARNER]: Advancing to term 11
I20260709 12:18:52.245683 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:52.223804 23881 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.250909 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:52.251816 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.239951 23816 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:52.264629 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90108.5us
I20260709 12:18:52.266795 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.249718 23820 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:52.285394 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.262735 23750 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:52.287606 23896 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.292833 19043 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:18:52.293092 19036 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:52.291220 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:52.298761 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 113682us
I20260709 12:18:52.286439 23697 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:52.273352 23907 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.284880 23820 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:52.307343 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.280064 23911 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.301925 23697 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:52.303746 23519 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.319640 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.323508 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.329668 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.302754 23750 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.318193 23697 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.314503 23820 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.324071 23541 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.326251 23519 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.340341 23519 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.343945 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.344646 18443 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 9.
I20260709 12:18:52.169873 18192 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 6, term changed from 8 to 10, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65), NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 10 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:18:52.346053 18192 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 16, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 16 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:52.347797 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.348013 23909 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:52.347092 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.347646 18192 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 12, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:52.349483 23750 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.350333 23541 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:52.350127 23905 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:52.336359 23911 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.353565 23816 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.368806 18375 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:52.313990 23907 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:52.356149 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 122073us
W20260709 12:18:52.358857 23914 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:18:52.359275 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.891s (SENT) [suppressed 56 similar messages]
I20260709 12:18:52.373247 23907 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.377278 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:52.377962 18446 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:52.355087 23906 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:52.380791 18443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:46956
I20260709 12:18:52.381371 18443 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:52.381814 18443 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.363952 23722 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.384375 19039 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:18:52.388135 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.393148 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.384718 19039 pending_rounds.cc:85] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 13
W20260709 12:18:52.394582 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71194.7us
I20260709 12:18:52.394801 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.395360 19040 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.395330 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.433303 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.433740 18444 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:52.434180 19035 raft_consensus.cc:2410] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:18:52.438649 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 113506us
I20260709 12:18:52.440162 23820 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.441159 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.386233 23822 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.442636 23820 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.444993 23697 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.352864 23826 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.448496 23897 raft_consensus.cc:993] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:52.449091 23897 raft_consensus.cc:1081] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:18:52.440156 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.454876 23682 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:52.455425 23627 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.456445 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.394338 19039 pending_rounds.cc:107] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:18:52.464933 19039 raft_consensus.cc:2889] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.465035 18442 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:46956
W20260709 12:18:52.457489 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 59452.1us
I20260709 12:18:52.465571 18442 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:52.465935 18442 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.352867 23541 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.457321 19040 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:52.467195 18372 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:52.467819 18372 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.446996 23697 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.470166 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:52.468953 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65807.3us
I20260709 12:18:52.477351 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.456353 18192 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 15, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:52.432880 23907 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.443193 23820 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:52.490125 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 239298us
I20260709 12:18:52.470559 23541 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.394084 23525 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:18:52.489847 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:52.509660 18367 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:52.516245 23919 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:52.516953 23919 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.464433 23895 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:18:52.520358 23895 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:52.520725 23895 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.524482 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.525166 18368 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:52.525563 23826 raft_consensus.cc:993] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:52.526072 23714 raft_consensus.cc:1081] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:18:52.467854 23894 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.527545 23894 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.529877 23873 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.470185 23682 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.534754 23872 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.535485 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.498112 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.498791 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.499471 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.501542 19040 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:52.502312 18442 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.537485 19043 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:52.507524 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.539649 18366 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:52.494987 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:52.542536 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 107799us
I20260709 12:18:52.479319 23911 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.546396 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.539852 18372 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.549894 18368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:18:52.550663 18368 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:52.551136 18368 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:52.551792 18448 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:52.493952 23896 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:52.552603 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.553594 19040 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:52.554615 18442 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.478266 23722 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:18:52.558516 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:52.594867 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.584971 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.529891 23894 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:52.562551 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.541836 23816 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.471697 23627 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.542479 23753 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.521656 23895 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.553164 23907 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.546640 23682 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.616122 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:52.618095 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 120062us
I20260709 12:18:52.618340 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.619827 18291 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:52.620576 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.621157 18299 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:52.621356 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:18:52.622557 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 132501us
I20260709 12:18:52.621990 19035 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:52.621961 18291 raft_consensus.cc:2376] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12. Re-sending same reply.
I20260709 12:18:52.624008 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.556728 23896 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.624755 18295 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:52.607062 23920 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:52.614394 23872 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:52.552908 23911 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.638721 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.639168 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.639710 23920 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:52.640195 23911 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.549211 23820 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.640662 23920 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.639189 23894 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.642861 18366 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:52.642697 23894 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.643738 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.635452 18192 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:52.639375 18370 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:52.644241 23884 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.638207 23627 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.645412 18444 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:52.645412 18192 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 11, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:18:52.646219 23884 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.601588 23721 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:18:52.612178 23921 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.652946 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.619581 23919 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.656311 18442 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:52.668154 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.668840 18299 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:52.668040 18372 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.655211 23894 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.671260 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.696210 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.696959 18299 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:52.626549 23896 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.642727 23820 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.697573 18443 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.696318 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:52.699198 18449 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.699746 18295 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:52.699667 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141221us
I20260709 12:18:52.650972 23907 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.704187 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.705127 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.708155 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.708887 23894 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:18:52.703079 23884 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.711850 18443 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.679615 23921 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:52.715821 23924 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:18:52.707027 23721 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:52.698803 23826 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:18:52.717218 23826 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:52.717582 23826 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.718438 23826 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.647133 18192 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:52.723470 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 94021.5us
I20260709 12:18:52.724617 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.726655 23697 raft_consensus.cc:2764] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:18:52.715090 23921 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.701079 23919 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.732317 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.740548 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.741178 19043 raft_consensus.cc:2376] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13. Re-sending same reply.
I20260709 12:18:52.739660 18295 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:52.736855 18368 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.744185 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:52.751475 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 98603.4us
I20260709 12:18:52.754063 18368 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.703703 23541 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.741044 23911 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:52.766747 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.768777 23907 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:52.744882 23721 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:18:52.779798 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:52.780498 18292 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.748979 23894 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 12:18:52.789829 23923 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:18:52.775632 23826 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.810449 23921 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.811154 23921 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.813946 23921 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.817593 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.818538 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.820572 23911 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:52.820950 23911 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.821282 23911 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.824527 23873 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:52.825978 23928 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:52.829392 23721 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.829939 23721 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.830525 23911 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.832324 23541 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.835014 23895 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.844303 18295 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:52.845651 23911 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:52.845976 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52139.7us
I20260709 12:18:52.852463 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.853714 23873 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:52.854045 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.854321 23873 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:52.855527 23873 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.857700 23682 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:52.858002 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.858630 18367 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:52.858959 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:52.861256 23750 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.861667 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:52.863957 23883 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:52.864351 23883 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.864681 23659 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.865921 23894 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.861693 23750 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.869128 23750 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.869872 23750 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:52.864701 23883 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.870987 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.871583 18370 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:52.874667 18367 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:52.875527 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:52.876067 18366 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.876797 18446 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:52.877228 18446 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:52.877961 18446 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.878880 18446 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.874951 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.881348 23682 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:52.881973 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:52.882398 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.883261 23627 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:52.883639 23627 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.883993 23627 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.889458 18366 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:52.890600 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.892133 18292 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:52.884666 23721 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:52.907877 18299 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:52.893237 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:52.893891 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 167778us
I20260709 12:18:52.908051 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:52.908658 18449 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.901522 23714 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:52.903901 18366 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:52.908717 18367 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:52.911440 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.913825 18412 leader_election.cc:400] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
I20260709 12:18:52.915369 18412 leader_election.cc:403] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:52.917260 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.920141 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.920321 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.922649 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:52.923357 18370 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.921130 18295 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 11.
I20260709 12:18:52.927541 18368 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:52.935781 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.936658 18263 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.939687 18449 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:52.926744 23907 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.939592 23541 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.945901 23907 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.946482 23541 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:52.948163 23907 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.947945 23136 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.949200 23136 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:52.950345 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.948745 23541 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:52.949587 23136 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.958822 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.961155 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.962358 19036 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.955961 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:52.963569 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.964310 19039 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:52.965104 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.966236 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:52.963585 18449 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:52.967893 23525 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:52.967418 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:52.969350 19041 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:52.907810 23822 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:18:52.969645 19041 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.972141 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:52.973624 23136 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.974117 23541 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:52.974566 23541 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.974809 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.975651 18295 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:52.977169 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.920964 23884 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13.
I20260709 12:18:52.927017 23721 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:52.922173 23821 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.979887 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:52.984263 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:52.984750 18291 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.985136 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:52.982060 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.982342 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:52.986840 18368 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 10.
I20260709 12:18:52.974970 23541 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:52.970660 19041 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:52.908733 23659 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:18:52.993448 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:52.994463 18292 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:52.932132 23722 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:52.988956 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:53.004109 19040 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:52.986186 18367 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:52.990696 19041 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.991992 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:52.992170 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:53.008910 18294 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.012130 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.012990 18298 raft_consensus.cc:2368] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
I20260709 12:18:53.016098 18297 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:53.018692 23883 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.020087 19035 raft_consensus.cc:2368] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14.
I20260709 12:18:53.020991 23883 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:52.995635 23668 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:53.023818 23668 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.024622 23668 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.025946 23668 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:53.027683 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49153.7us
I20260709 12:18:53.028494 23659 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:53.028918 23659 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:53.029299 23659 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:53.029443 23897 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:53.029927 23897 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:52.976780 23928 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:53.032357 18338 connection.cc:256] RPC call timeout handler was delayed by -1.2975742460000674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 12:18:53.032231 23928 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:53.032748 18338 connection.cc:256] RPC call timeout handler was delayed by -1.265174246000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 12:18:53.032783 23697 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.918377 23750 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.033399 23697 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.035389 23931 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.035774 23682 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:53.036980 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.037674 18300 raft_consensus.cc:2376] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 10. Re-sending same reply.
I20260709 12:18:53.038741 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.039650 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:53.042604 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 95041.5us
I20260709 12:18:53.043627 18446 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:18:53.044634 18446 pending_rounds.cc:85] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:18:53.044991 18446 pending_rounds.cc:107] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:18:53.045317 18446 raft_consensus.cc:2889] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.981329 23919 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:53.046551 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54503.8us
I20260709 12:18:53.046074 19041 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:53.047153 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.048954 18293 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:53.049165 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:53.049849 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:53.046418 18294 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:53.053589 18370 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:52.992761 23926 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.932991 23895 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 12:18:53.056273 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.067504 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.068264 18297 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.068917 18293 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:53.063165 18291 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:53.003736 23722 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.035878 23931 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.074756 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.078840 18368 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.043361 23682 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:53.030802 23897 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.079495 23682 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.080447 23897 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.066133 23926 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:53.087716 23926 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.088248 23714 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:53.088693 23714 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.089047 23714 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.089075 23937 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:53.046270 23750 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.055747 23721 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:53.108171 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.089666 23937 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.123940 18368 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:53.063834 23697 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.077802 23722 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:53.127069 18370 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:53.128208 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.048249 23820 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:18:53.125787 23937 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.143847 23721 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.099891 23940 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:53.152298 23627 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:53.152194 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.116425 23919 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:53.158911 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92569.2us
W20260709 12:18:53.154577 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 121210us
I20260709 12:18:53.172423 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:53.176808 19041 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:53.177875 23697 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:18:53.182591 18370 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:53.156605 23940 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:53.186146 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 136429us
I20260709 12:18:53.162921 23753 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:53.188236 19006 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:53.187521 23822 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:53.192673 23822 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.194181 23822 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.192085 23753 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:53.189275 23940 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:53.198699 23942 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.190111 23826 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:18:53.190871 23919 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:53.185179 23897 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:18:53.211104 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.212396 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:53.215510 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.216238 18375 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:53.220232 19038 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:53.255210 23627 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.213169 23942 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:53.256341 23942 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.258905 23919 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.264016 23682 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.266587 23682 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:53.267784 23934 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:18:53.269559 23939 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:18:53.285614 23722 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.204160 23896 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.289484 23896 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.291407 23753 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.293282 23753 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.300925 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.324863 18368 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:53.311932 23541 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:53.348243 23943 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:53.353801 18368 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:18:53.357012 18375 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:53.360752 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:53.366182 23714 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.368772 23714 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:53.369812 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.374289 23541 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.376017 23525 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.376513 23525 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.378468 23525 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.378520 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:53.382570 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.383112 18449 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.384502 18293 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:53.387817 18370 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:53.387820 18294 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:18:53.389868 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.390704 18370 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:53.390652 18297 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:53.392725 23136 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.395078 19036 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:53.395583 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.396154 18448 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:53.396191 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.397670 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:53.398353 19039 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:53.399565 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.399926 23931 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.400096 19037 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:53.400508 19037 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.400607 23920 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.401264 19037 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.401316 23920 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.402351 19037 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:53.404393 23948 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 12:18:53.405604 18291 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:53.399192 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.406811 18295 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:18:53.407110 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.407717 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:53.409595 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.411135 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.411978 18291 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:53.413306 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.413553 18299 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:53.415201 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.415550 18299 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:53.415758 18295 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:18:53.420918 18449 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:53.422488 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:53.423097 18300 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:53.424491 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.425036 18370 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:53.429334 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.431911 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.432575 18293 raft_consensus.cc:2368] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:53.432693 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.433308 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.433851 18294 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:53.435788 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.436364 18366 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.438423 18368 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:53.438956 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.439345 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.439888 18375 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.440678 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.444348 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.444885 18293 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.450470 18294 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:18:53.451727 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.453141 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.459767 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.462548 18441 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:53.456358 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74017.6us
I20260709 12:18:53.461959 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.467921 18373 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:53.469497 19036 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:53.471427 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.477025 19039 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:53.479668 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:53.427793 23894 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:53.481609 18367 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:53.484735 18370 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:53.437079 23821 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.453437 23519 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.487884 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.486972 18300 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:53.493784 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:53.457315 23884 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.495566 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.496124 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.511211 19039 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:53.504391 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.505023 23821 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.474911 23911 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.499758 18293 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:53.500674 23938 raft_consensus.cc:1064] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:18:53.514317 18298 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:53.517998 19036 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:53.520741 18366 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
W20260709 12:18:53.526935 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.475059 23907 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:53.522128 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58615.7us
I20260709 12:18:53.456097 23822 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:18:53.542219 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.548974 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:53.549682 23947 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.214s	user 0.002s	sys 0.000s
I20260709 12:18:53.552405 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.553557 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:53.531468 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:53.533460 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.513149 23750 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:53.574433 23750 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:53.574909 23750 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:53.573134 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.577128 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.578258 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.578872 18293 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:53.581185 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.523490 23920 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.587622 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:53.588557 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.599004 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.511992 23519 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.586601 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:53.558140 23136 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:53.522194 23884 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:53.618227 18339 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.887s (SENT)
I20260709 12:18:53.620855 19037 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:53.617622 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.632825 18367 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.634889 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:53.583673 23826 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.636679 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.554637 23895 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:18:53.639320 18441 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:53.556710 23907 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.514950 23659 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.646270 18293 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:53.649471 18449 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:53.545841 23911 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:53.649657 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.653208 18366 raft_consensus.cc:2368] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:53.653564 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.653970 19037 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:18:53.618978 23894 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:53.637717 23714 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.633404 23928 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.637846 23668 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:53.656880 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:18:53.696419 18293 raft_consensus.cc:606] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:53.696751 18293 raft_consensus.cc:614] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:53.655265 18339 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.531s (SENT)
I20260709 12:18:53.697358 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:53.698159 23931 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:53.694388 19038 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:53.698998 23931 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:53.700352 23931 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.702880 23931 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.658418 18441 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:53.695851 18367 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:53.715374 19037 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:53.715741 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.719878 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.722456 19037 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:53.763171 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.763989 18367 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.764925 23971 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:53.772336 23940 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 12:18:53.772055 23971 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.772728 23940 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.774650 23971 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.775456 19035 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:53.775427 18373 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:18:53.776253 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.712076 23928 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.779050 23928 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.779345 23896 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:53.779873 23896 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.781339 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.781790 23896 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.782536 23839 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.782956 18446 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:53.782946 18367 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:53.784082 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.784288 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:53.771597 23955 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:18:53.774724 23940 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:53.742550 23541 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.787127 18375 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:53.782910 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.742623 23970 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.793608 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.805569 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.806515 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:53.806820 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:53.771023 23926 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.000s
I20260709 12:18:53.826350 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:53.794075 18448 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:53.783589 18446 pending_rounds.cc:85] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:18:53.828487 18446 pending_rounds.cc:107] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:53.828827 18446 raft_consensus.cc:2889] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.802817 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.717015 23668 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.821451 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:53.832818 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 198024us
I20260709 12:18:53.836978 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:53.842060 23921 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:53.844329 23541 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:53.844877 23541 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:53.844980 23907 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.845754 23541 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.848466 19042 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:53.849757 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:53.851514 23936 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:18:53.851994 23936 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:53.852873 23936 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.856956 23973 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:53.857473 23973 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.857925 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.858510 18375 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:53.859311 23519 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.808169 23894 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.863977 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:53.864691 23952 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.508s	user 0.000s	sys 0.002s
I20260709 12:18:53.868583 23884 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.792471 23839 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.883903 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.888602 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:53.905862 18375 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:18:53.906036 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.808521 23821 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.916953 18367 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:53.918869 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.919946 18373 raft_consensus.cc:2410] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:18:53.923980 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:53.928792 18372 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:53.913453 23955 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:18:53.804903 23920 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 12:18:53.946084 19041 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:53.907171 23940 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.952111 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:53.910560 23714 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.920367 23964 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:53.944765 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 138367us
I20260709 12:18:53.960630 23714 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:53.961391 23966 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:18:53.962059 23659 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.919129 23826 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.963546 23911 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.964039 23826 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:53.965974 23942 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.709s	user 0.005s	sys 0.003s
I20260709 12:18:53.966251 23973 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.966398 23942 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.958974 23722 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:53.967375 23883 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:53.967846 23883 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:53.968644 23668 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:53.968989 23883 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:53.970939 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:53.963575 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:53.973052 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.957156 23937 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.798s	user 0.010s	sys 0.000s
I20260709 12:18:53.957815 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:53.975584 18372 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:53.839525 23926 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.973090 19043 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:53.972313 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:53.982894 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:53.974160 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:53.988864 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.973845 23519 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.993955 23839 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:53.996151 23753 consensus_meta.cc:287] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
I20260709 12:18:53.825587 23970 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:53.959486 23964 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.005198 19039 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:54.008299 18446 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:54.009415 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:54.016160 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.016826 18443 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:54.010404 18292 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:54.011828 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:54.013898 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:53.953639 23940 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.018500 18449 raft_consensus.cc:2368] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16.
I20260709 12:18:54.051615 18299 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:54.007614 23136 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:54.019143 23964 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.055374 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:54.063706 18448 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:54.065527 18448 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:53.987198 23937 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:54.064204 19042 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
W20260709 12:18:54.054409 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 431513us
I20260709 12:18:54.066322 18448 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.077998 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.078476 18448 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.051350 23753 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:54.080448 18295 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:54.083312 23937 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.084162 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.084903 18447 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:54.054803 23839 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:54.087594 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:54.088554 18372 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:54.089048 23942 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.090646 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:54.091347 18372 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.092141 23750 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.092969 18295 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:54.094136 23750 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.095417 18443 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:18:54.097992 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.098654 18442 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:18:54.099408 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44149.6us
I20260709 12:18:54.100147 19007 leader_election.cc:400] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16.
I20260709 12:18:54.100427 19007 leader_election.cc:403] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:18:54.101541 23977 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:18:54.102401 23972 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:54.102852 23972 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.103008 23627 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.103493 23944 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.103978 23944 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:54.104336 23944 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.105578 23972 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:54.110008 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.112864 23627 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16.
I20260709 12:18:54.113845 23942 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.122052 23942 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.114661 23895 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.124814 23926 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.125218 23895 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.128787 23970 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.129647 19036 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:54.115599 23753 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:54.118157 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:54.130889 19044 raft_consensus.cc:606] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:54.131338 19044 raft_consensus.cc:614] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.116444 23136 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.156270 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:54.122479 23964 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:54.118379 23940 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.132690 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:54.135835 18296 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:54.141987 18372 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11.
I20260709 12:18:54.011432 23519 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:54.173885 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:54.174242 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:54.174497 18367 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:18:54.102926 23967 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:18:54.121179 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:54.168699 18295 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:54.169638 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.171653 18448 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:54.160893 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:54.209132 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:54.209805 18443 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:54.210997 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.212806 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.213217 18294 raft_consensus.cc:2393] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:54.214008 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92927.8us
W20260709 12:18:54.215219 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54468.1us
I20260709 12:18:54.222916 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:54.209606 23951 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.817s	user 0.005s	sys 0.000s
I20260709 12:18:54.173722 23820 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.228291 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 218884us
I20260709 12:18:54.207209 23921 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:54.211248 23942 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.229631 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:54.221803 23714 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:54.115180 23972 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.253713 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:54.234807 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:54.238118 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.288960 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:54.265132 18338 leader_election.cc:400] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
W20260709 12:18:54.264410 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 290326us
I20260709 12:18:54.289906 18372 raft_consensus.cc:2410] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 12:18:54.234314 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 409487us
I20260709 12:18:54.306070 18373 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:54.227949 23968 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.307472 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:54.306078 18338 leader_election.cc:403] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:54.310669 18295 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:18:54.313252 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48266.9us
I20260709 12:18:54.224455 23872 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.317786 19044 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d266f68528964b1981ffe1df49861b08"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:54.318524 19044 raft_consensus.cc:606] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:54.318921 19044 raft_consensus.cc:614] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.236474 23895 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.322036 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.323024 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:54.323364 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 68222.9us
I20260709 12:18:54.324770 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:54.326084 19043 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:54.326817 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:54.326900 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:54.327715 19043 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:54.328497 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:54.329526 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.231603 23936 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:18:54.330143 18295 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:54.340176 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:54.317941 23921 raft_consensus.cc:2764] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:18:54.313527 23968 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:54.318202 23753 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
I20260709 12:18:54.238958 23820 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.354676 18295 raft_consensus.cc:2393] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:54.347877 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:54.323500 23714 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:54.367856 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 112765us
I20260709 12:18:54.371680 19043 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:18:54.375453 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:54.327818 23872 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.331247 23983 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:54.333657 23884 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:18:54.354115 23968 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.352824 23622 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:54.336935 23982 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:18:54.383704 23753 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.435370 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:18:54.437801 19043 raft_consensus.cc:606] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:54.414088 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44262.6us
I20260709 12:18:54.441797 19043 raft_consensus.cc:614] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:54.428025 24001 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 12:18:54.452689 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.382266 23960 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 12:18:54.436753 23926 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:54.442417 23921 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:54.443899 23964 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:54.457443 23519 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:54.450541 23972 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:54.589612 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.595973 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 274777us
I20260709 12:18:54.597168 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:54.598363 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:54.438727 23986 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 12:18:54.615180 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:18:54.593694 23992 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:18:54.619578 23985 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.621840 23987 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:18:54.622035 23985 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.622810 23996 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:54.600440 23980 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:18:54.428925 23714 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.599310 23753 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.471356 23926 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:54.623428 23956 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.058s	user 0.008s	sys 0.009s
I20260709 12:18:54.601890 23964 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:54.594823 23816 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:54.636456 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:54.636837 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 170953us
I20260709 12:18:54.637276 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:54.621280 23921 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:54.640630 23995 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:54.647188 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:54.624014 23982 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:54.638250 23983 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:54.641697 23820 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:54.706154 18415 leader_election.cc:400] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:54.706619 18415 leader_election.cc:403] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:18:54.696412 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 59219.2us
I20260709 12:18:54.746603 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:54.637828 23960 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:18:54.642609 23816 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.637734 23622 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:18:54.636514 23753 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:54.649727 23970 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:54.765156 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 69935us
W20260709 12:18:54.784899 23980 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:18:54.723308 23998 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.231s	user 0.000s	sys 0.008s
I20260709 12:18:54.644348 23964 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:54.700645 23872 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:54.868247 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:18:54.832408 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.870507 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.815721 23975 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:54.817790 23816 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.879109 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:54.978112 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1253) took 1766 ms. Trace:
W20260709 12:18:54.981153 18339 leader_election.cc:336] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.159s (SENT)
I20260709 12:18:54.783007 23921 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:54.981343 19040 rpcz_store.cc:276] 0709 12:18:53.211812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.211931 (+   119us) service_pool.cc:224] Handling call
0709 12:18:53.212287 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.304188 (+ 91901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:53.304234 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:53.308216 (+  3982us) log.cc:844] Serialized 26 byte log entry
0709 12:18:53.411444 (+103228us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:53.411539 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.411605 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.623311 (+1211706us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b20001be5f8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68a2ab91 00007f3c68a4ca19 00007f3c68a178c3 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169 00007f3c68a20131 00007f3c68a1f42d 0000000000388634
0709 12:18:54.876101 (+252790us) raft_consensus.cc:1685] finished
0709 12:18:54.876119 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.978015 (+101896us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1295,"raft.queue_time_us":5654,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":1583,"spinlock_wait_cycles":11837184,"thread_start_us":81728,"threads_started":1,"wal-append.queue_time_us":171042}
I20260709 12:18:54.868697 23983 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:54.817138 23988 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:54.989974 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1055) took 1451 ms. Trace:
I20260709 12:18:54.992594 23622 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:54.993333 18298 rpcz_store.cc:276] 0709 12:18:53.539066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.539352 (+   286us) service_pool.cc:224] Handling call
0709 12:18:53.539659 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.540047 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:53.540088 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:53.566598 (+ 26510us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.015602 (+449004us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:54.052140 (+ 36538us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.052529 (+   389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.052617 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:54.989694 (+937077us) raft_consensus.cc:1685] finished
0709 12:18:54.989724 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:54.989888 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13551,"prepare.queue_time_us":33381,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2084,"raft.queue_time_us":19900,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":15679,"spinlock_wait_cycles":26111360,"thread_start_us":395821,"threads_started":1,"wal-append.queue_time_us":635377}
I20260709 12:18:54.981894 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:55.003453 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 256948us
I20260709 12:18:54.980672 23820 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:55.029167 23873 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:55.033430 24010 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:55.033365 23873 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.034194 24010 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.008474 23622 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.035624 24006 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.394s
I20260709 12:18:55.035243 23873 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:54.782737 24004 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.341s
I20260709 12:18:55.037262 24004 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:55.037667 24004 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:54.995960 24003 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:55.040755 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 47280.1us
I20260709 12:18:54.996167 23999 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.039129 24013 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:55.037220 23622 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.045187 24006 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:55.054235 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.012s	sys 0.044s
W20260709 12:18:55.056569 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.012s	sys 0.046s
W20260709 12:18:55.049553 23994 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.555s	user 0.001s	sys 0.001s
I20260709 12:18:55.032915 24011 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:55.041049 24003 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:55.045163 24005 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.420s	user 0.001s	sys 0.000s
W20260709 12:18:55.053315 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 174275us
W20260709 12:18:55.048935 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 112077us
I20260709 12:18:55.053467 23995 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:54.995968 23988 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.061709 24003 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.062808 24003 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.062870 24016 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:55.063359 23988 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.063762 24016 consensus_queue.cc:1243] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:18:55.064891 23995 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.057782 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 699241us
I20260709 12:18:55.052670 23985 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:54.981071 23975 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.039952 23820 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.070946 23953 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.649s	user 0.007s	sys 0.007s
I20260709 12:18:55.071310 23753 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.073565 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1054) took 1571 ms. Trace:
W20260709 12:18:55.068785 24008 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:18:55.076263 23975 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.067744 23926 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.058419 23921 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.056563 24013 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:55.080900 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:55.035175 24010 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.047816 23999 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.073568 24007 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.038494 24004 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.097467 23999 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:55.097601 23944 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.991s	user 0.007s	sys 0.004s
I20260709 12:18:55.097733 24004 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.098093 23944 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.058631 23983 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.088049 24018 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:55.101853 24018 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:18:55.102233 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.089486 23964 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.088220 23753 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:18:55.091068 24013 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.094112 23968 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.000s
I20260709 12:18:55.084578 23985 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 12:18:55.097976 23622 consensus_queue.cc:1175] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e23c0ce8d89940da8eac6a6002e349c5" responder_term: 12 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:55.082996 18300 rpcz_store.cc:276] 0709 12:18:53.502245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.503492 (+  1247us) service_pool.cc:224] Handling call
0709 12:18:53.503798 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:53.994123 (+490325us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b18003aee20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e74bfb 00007f3c68e7f64b
0709 12:18:53.994303 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:53.994359 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:53.995080 (+   721us) log.cc:844] Serialized 52 byte log entry
0709 12:18:54.246461 (+251381us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:54.246511 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.246600 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.073252 (+826652us) raft_consensus.cc:1685] finished
0709 12:18:55.073272 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.073498 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4737,"raft.queue_time_us":40408,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":11495,"spinlock_wait_cycles":33450880,"thread_start_us":299291,"threads_started":2,"wal-append.queue_time_us":387697}
I20260709 12:18:55.059166 24006 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:55.110394 18412 leader_election.cc:400] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:18:55.139472 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:55.110769 24015 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:55.112802 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.141706 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.113130 24019 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.112414 23938 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.115286 23970 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:55.118685 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.088776 24020 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:18:55.123463 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2239) took 1237 ms. Trace:
I20260709 12:18:55.146879 18299 rpcz_store.cc:276] 0709 12:18:53.885720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.886111 (+   391us) service_pool.cc:224] Handling call
0709 12:18:53.886366 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.052019 (+165653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.123165 (+1071146us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54000a9bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:55.123390 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379218688}
W20260709 12:18:55.131398 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61252.1us
I20260709 12:18:55.124454 23926 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.102726 24018 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.155540 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.176662 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:55.178893 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:55.179525 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.183977 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.141296 24015 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.148650 24019 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.179831 23985 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:18:55.201305 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.202080 18338 leader_election.cc:400] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14.
I20260709 12:18:55.202395 18338 leader_election.cc:403] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:55.202953 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.204001 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:55.204515 18447 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:55.206082 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:55.206924 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.190809 24018 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:55.260226 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.134151 23968 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.148352 24011 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 12:18:55.314035 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 112075us
I20260709 12:18:55.317317 23944 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.210414 23921 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.320465 24006 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:55.199385 24015 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.324136 23941 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.326647 23820 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.200811 24020 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.179713 23926 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.212291 23985 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.333014 23936 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.470824 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.416085 23919 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14.
I20260709 12:18:55.417735 18445 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:18:55.424183 18447 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:55.469627 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.314374 23873 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:55.361832 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.184597 23988 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:18:55.449290 23896 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:55.472738 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 239340us
I20260709 12:18:55.484105 23896 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.485366 18339 leader_election.cc:336] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.875s (SENT)
I20260709 12:18:55.474469 19040 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:55.486500 23896 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.475068 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:55.475893 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.490607 18372 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:55.476850 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.491544 18370 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:55.492358 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.492810 18370 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:55.477311 18373 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:18:55.478319 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:55.479439 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.479517 18367 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:55.480456 23873 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.480767 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.493875 18371 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:55.496294 18372 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:18:55.497845 18372 pending_rounds.cc:85] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 11
I20260709 12:18:55.498088 18372 pending_rounds.cc:107] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:18:55.498322 18372 raft_consensus.cc:2889] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:55.473821 23975 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.476857 23944 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.476701 23820 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.498976 23988 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:55.515398 23896 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.515982 23896 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.516553 23990 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.185s	user 0.033s	sys 0.005s
I20260709 12:18:55.518426 23896 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.475370 23936 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.521036 23936 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.466272 23941 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.523089 23941 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.524281 24015 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.527311 24015 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:55.508559 23975 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.529570 23975 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.510774 23944 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:55.532061 23873 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.532116 23921 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:55.534017 23944 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.509809 23820 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.536476 23926 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.537204 23985 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.476302 24020 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:55.544010 24019 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.544679 24020 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.501636 23816 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.547645 23816 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:55.548808 23816 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.549340 23816 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.551445 23816 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.552114 23988 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:55.553082 23988 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.559909 23964 raft_consensus.cc:1027] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: attempt to promote peer 991dcd2a70c74fdb89ca9494cc09fb7b: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:18:55.567252 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.567889 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.568260 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.539096 23820 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.576889 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:55.601052 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.605137 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.605998 18441 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:55.614269 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:55.616933 23714 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:55.620693 23714 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:55.621152 23714 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:55.644217 19040 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
W20260709 12:18:55.644218 19036 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 951) took 2250 ms (client timeout 2615 ms). Trace:
W20260709 12:18:55.645012 19036 rpcz_store.cc:269] 0709 12:18:53.393815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.525274 (+131459us) service_pool.cc:224] Handling call
0709 12:18:53.553429 (+ 28155us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:55.630818 (+2077389us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540055e6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:55.644134 (+ 13316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353573504}
I20260709 12:18:55.655445 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.657132 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.657766 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.657946 18299 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:55.658259 18293 raft_consensus.cc:1217] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Deduplicated request from leader. Original: 11.10->[15.11-15.11]   Dedup: 15.11->[]
I20260709 12:18:55.658752 18293 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:18:55.659624 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 942) took 2265 ms. Trace:
I20260709 12:18:55.660022 19038 rpcz_store.cc:276] 0709 12:18:53.379839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.381017 (+  1178us) service_pool.cc:224] Handling call
0709 12:18:53.381263 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.699522 (+318259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.631373 (+1931851us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400693054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:55.645245 (+ 13872us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657966336}
I20260709 12:18:55.661839 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.662447 18441 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:55.663164 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:55.663815 18298 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.663625 18443 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:55.667338 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.668004 18449 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:18:55.669925 18294 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:55.670019 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.670645 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:55.670874 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.663570 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.670934 18295 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:55.671489 18294 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:55.661921 23962 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.672099 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.672254 23962 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.672686 18295 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:55.674620 23962 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.674736 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.675315 18440 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:55.675830 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2476) took 1578 ms. Trace:
I20260709 12:18:55.676167 18440 rpcz_store.cc:276] 0709 12:18:54.097023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.100179 (+  3156us) service_pool.cc:224] Handling call
0709 12:18:55.674598 (+1574419us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:55.675751 (+  1153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24592768}
I20260709 12:18:55.677749 23975 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:55.678181 23975 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.678570 23975 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:55.682369 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.046s	sys 0.063s
W20260709 12:18:55.682745 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.046s	sys 0.063s
I20260709 12:18:55.683854 18441 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:55.675459 18299 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:55.672756 18294 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:55.687057 18298 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:55.688500 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:55.688591 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 549178us
I20260709 12:18:55.690037 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.645331 23820 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.691738 18441 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:55.655020 23682 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:55.694260 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.655283 23722 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:55.695199 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.683984 23938 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.701648 18441 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:55.699929 23682 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.688585 23896 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.702544 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.693095 23942 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.713194 23682 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:55.719293 23682 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:55.720259 24019 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:55.720649 24019 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.721504 24019 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.722728 24019 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:55.723618 18292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2609) took 2689 ms (client timeout 2648 ms). Trace:
W20260709 12:18:55.724079 18292 rpcz_store.cc:269] 0709 12:18:53.034645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.050775 (+ 16130us) service_pool.cc:224] Handling call
0709 12:18:55.723393 (+2672618us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54002369d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:55.723544 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166745856}
I20260709 12:18:55.725178 24013 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.725747 24013 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.728096 24013 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.730497 18297 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.708261 24004 raft_consensus.cc:2764] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:18:55.705179 23816 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:55.732650 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2225) took 2347 ms. Trace:
I20260709 12:18:55.704967 23721 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:18:55.733009 18297 rpcz_store.cc:276] 0709 12:18:53.385283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.412989 (+ 27706us) service_pool.cc:224] Handling call
0709 12:18:55.730316 (+2317327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.730889 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.732474 (+  1585us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54002369d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:55.732597 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5328646912}
I20260709 12:18:55.735146 24020 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.735769 24020 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.737644 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.738636 23816 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:55.739079 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.697260 23872 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:18:55.742002 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.742888 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.744901 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.701097 23820 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.703636 23722 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.752048 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.738269 24020 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.709059 23896 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.705750 23938 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.739641 23816 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.744627 23936 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:55.763350 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:55.714663 23942 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:55.765198 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 62022.3us
I20260709 12:18:55.757768 24004 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:55.774761 18339 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.760733 23682 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:18:55.760138 23884 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:55.793354 23884 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.794423 23722 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.794248 23884 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.794938 24026 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:55.795375 24026 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:55.796192 24026 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.775653 23873 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:55.772720 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.797394 24026 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:55.797801 23931 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.798177 23931 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:55.798622 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:55.798956 23931 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.800173 23931 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.800690 23926 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:55.801120 23926 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:55.801533 23926 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.788494 24013 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:55.789402 23959 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:18:55.787604 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:55.803735 23959 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:55.804152 23959 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.800086 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.805003 23959 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.806186 23959 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.807598 23944 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:55.808364 23944 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:55.811529 23541 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:18:55.812603 23964 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:55.812686 23896 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.813148 23964 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:55.813639 23964 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.814723 23820 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.817441 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:55.818200 18297 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:55.819056 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:55.813067 23541 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:55.820035 23541 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.820432 23541 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:55.820919 23821 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:55.823071 24020 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:55.761320 24006 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:18:55.823647 24020 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.823925 23820 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:55.824909 23883 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.826480 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.792724 24004 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.829777 24004 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.826426 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.831597 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.832002 23820 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:55.832129 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.832480 23820 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:55.833069 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.832934 23820 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.834372 18298 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:55.834685 23944 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:55.835083 23944 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:55.835625 23944 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:55.839140 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.842620 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.844816 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.846330 18338 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:55.847571 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:55.849509 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.849723 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.850481 18297 raft_consensus.cc:2393] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:55.852900 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.838663 24013 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:55.852571 23873 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:55.858451 24013 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.865180 24013 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.858618 23919 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:55.866654 23919 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:55.856848 23921 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:18:55.867537 23820 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:55.868489 23938 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:55.869648 23820 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.870018 24020 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:55.871104 23989 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.549s	user 0.004s	sys 0.016s
I20260709 12:18:55.871753 23985 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:55.872159 23985 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.872366 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.867563 23919 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.873219 23985 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.874258 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.875645 18372 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:18:55.847487 23753 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.876667 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.877769 18370 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:18:55.877825 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2236) took 2443 ms. Trace:
I20260709 12:18:55.879098 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.879717 19043 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:55.881139 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.878268 18291 rpcz_store.cc:276] 0709 12:18:53.434140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.434268 (+   128us) service_pool.cc:224] Handling call
0709 12:18:53.434673 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:53.480615 (+ 45942us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:53.522961 (+ 42346us) log.cc:844] Serialized 14 byte log entry
0709 12:18:53.777947 (+254986us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b18004527a0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:18:54.016760 (+238813us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:54.048207 (+ 31447us) log.cc:844] Serialized 26 byte log entry
0709 12:18:54.048370 (+   163us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:54.065322 (+ 16952us) log.cc:844] Serialized 14 byte log entry
0709 12:18:54.065616 (+   294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.065705 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:55.870552 (+1804847us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:55.877641 (+  7089us) raft_consensus.cc:1685] finished
0709 12:18:55.877663 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:55.877774 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62199,"prepare.queue_time_us":60760,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":71,"raft.queue_time_us":57260,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":69448,"spinlock_wait_cycles":178746368,"thread_start_us":440170,"threads_started":1,"wal-append.queue_time_us":671607}
I20260709 12:18:55.882262 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.887825 18367 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:55.887912 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.888274 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.888494 19043 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:55.883873 18368 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:55.889411 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.892304 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.893745 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.894686 18366 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:55.895344 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.896358 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.898138 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:55.897912 23820 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:18:55.901732 18373 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:18:55.906064 24020 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.907752 19036 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.909106 24003 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:55.909492 24003 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:55.910333 24003 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:55.912683 19035 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 1252) took 2701 ms (client timeout 2056 ms). Trace:
W20260709 12:18:55.916139 19039 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 1258) took 2473 ms (client timeout 2178 ms). Trace:
I20260709 12:18:55.916991 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:55.917893 19037 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:55.916621 19039 rpcz_store.cc:269] 0709 12:18:53.443123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.522385 (+ 79262us) service_pool.cc:224] Handling call
0709 12:18:53.552251 (+ 29866us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5800113a98. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:55.915871 (+2363620us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54007e8bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:55.916054 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076162688}
I20260709 12:18:55.918968 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:55.839501 23936 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:55.927631 18445 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:55.913208 19035 rpcz_store.cc:269] 0709 12:18:53.211423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.211540 (+   117us) service_pool.cc:224] Handling call
0709 12:18:55.912396 (+2700856us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400478454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:55.912591 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275034880}
I20260709 12:18:55.924912 19041 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:18:55.882131 23946 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.925748 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.931545 18291 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:55.931799 18293 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:55.932093 24050 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.933231 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.937743 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.901310 23985 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.939214 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.941147 18293 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.899598 24016 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:55.954634 18295 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:55.939002 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:55.941622 24013 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:55.945514 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.947679 24050 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.949816 23938 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.952163 23975 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.953176 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.975522 18373 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:18:55.980481 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:55.981657 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:55.972635 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:55.925433 24003 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.928112 23820 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:55.978173 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.928684 23920 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.972635 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:55.900403 23714 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:55.989239 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:55.983611 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:18:55.985853 23986 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:55.988267 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:55.989916 18294 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:18:55.992570 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.953945 23896 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.997735 24016 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.004699 18292 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:56.014520 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.974032 23873 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:18:55.986450 23820 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:55.989547 23968 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:55.982757 23959 consensus_queue.cc:1059] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:18:55.978992 23960 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:18:56.025772 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1163) took 2070 ms (client timeout 1722 ms). Trace:
I20260709 12:18:56.028163 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:55.983064 24006 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:55.994503 23986 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:55.953699 23946 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.060606 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.009954 24016 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.986858 23872 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:55.986871 24010 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:18:56.080323 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1264) took 1446 ms. Trace:
I20260709 12:18:56.101222 24016 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:56.079267 19042 rpcz_store.cc:269] 0709 12:18:53.948516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.963309 (+ 14793us) service_pool.cc:224] Handling call
0709 12:18:56.018732 (+2055423us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b54008205d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:56.018854 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209856256}
I20260709 12:18:56.090232 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:55.986748 23938 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.094429 23986 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:56.027169 24025 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:56.002853 23920 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.013585 23896 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.028705 24019 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:18:56.104444 19041 rpcz_store.cc:276] 0709 12:18:54.634105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.634217 (+   112us) service_pool.cc:224] Handling call
0709 12:18:55.912556 (+1278339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.929340 (+ 16784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.080175 (+150835us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400478450. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:56.080275 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2939236736}
I20260709 12:18:56.128271 19040 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:56.026549 23541 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.173923 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.097034 23926 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.202708 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.139084 23942 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.204771 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.137156 24025 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.212173 23136 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:56.213968 23821 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:56.276849 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.165169 23820 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.275490 23959 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:56.280047 23722 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:56.286551 24055 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:18:56.301079 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.283671 24013 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.312060 18445 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:56.312321 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.318181 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2619) took 2158 ms. Trace:
I20260709 12:18:56.318753 18296 rpcz_store.cc:276] 0709 12:18:54.159841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.220821 (+ 60980us) service_pool.cc:224] Handling call
0709 12:18:54.233984 (+ 13163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.313128 (+ 79144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:54.313163 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:54.313205 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:54.313240 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.317863 (+2004623us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b5400275bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:56.317935 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.318107 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4710,"raft.queue_time_us":5184,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7665,"spinlock_wait_cycles":170380544}
I20260709 12:18:56.300470 23959 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:56.322989 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.309021 23921 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.305045 23714 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.282946 23821 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.281368 23136 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:56.334949 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.325404 18444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2479) took 2227 ms (client timeout 2786 ms). Trace:
I20260709 12:18:56.328747 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.344748 18445 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:18:56.312595 23975 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:56.338799 18444 rpcz_store.cc:269] 0709 12:18:54.097805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.100726 (+  2921us) service_pool.cc:224] Handling call
0709 12:18:56.322781 (+2222055us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:18:56.325347 (+  2566us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67837696}
I20260709 12:18:56.305423 23722 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.354472 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.326398 24071 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.361159 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.347397 23820 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:56.364225 24066 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:18:56.365265 23896 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:56.365567 24062 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:18:56.440083 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.441802 18374 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:18:56.370296 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 950) took 2976 ms. Trace:
I20260709 12:18:56.442441 19037 rpcz_store.cc:276] 0709 12:18:53.393581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.754155 (+360574us) service_pool.cc:224] Handling call
0709 12:18:55.917641 (+2163486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.918442 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.370063 (+451621us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54007e8bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:56.370232 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4974564096}
I20260709 12:18:56.372611 23926 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:56.373095 24061 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:18:56.349036 23946 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:56.430248 23931 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.011s
I20260709 12:18:56.429798 24071 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.452175 23722 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:56.358122 24065 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:18:56.373910 24057 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:18:56.357878 24016 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.420809 18445 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
W20260709 12:18:56.440227 24067 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.134s	user 0.005s	sys 0.002s
I20260709 12:18:56.435113 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.465081 18366 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:18:56.436791 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.437203 18375 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:56.437074 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.468922 18373 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:56.439437 18368 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:56.453188 18371 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:56.454960 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.360862 23938 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.362962 23920 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.431432 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.467258 23897 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.470899 18369 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:56.472090 18441 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:56.473688 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:56.454211 23753 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.572s	user 0.019s	sys 0.004s
I20260709 12:18:56.440089 23822 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:56.463109 23969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.013s
I20260709 12:18:56.476563 18445 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:56.345831 23921 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.454794 23714 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.452941 23750 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.452718 24069 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:56.480476 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.483201 18445 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:18:56.479394 23969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.003s	sys 0.013s
I20260709 12:18:56.476158 24025 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.488116 18445 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:56.479822 23753 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.490257 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.499429 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:56.500245 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.490382 23921 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.482738 24069 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:56.453536 23959 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.501968 23722 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.500691 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.504173 18445 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:56.507910 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.508988 18266 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.513860 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.481405 23822 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.521806 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.522593 18445 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.524374 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.527673 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.483373 23714 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.485564 23750 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:56.531344 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.534659 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.512328 23753 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.537727 23753 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.538060 24074 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:18:56.512265 23921 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.540199 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.547482 24076 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.547396 24069 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.548758 24069 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.550420 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.553061 23541 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.548784 23682 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:56.555713 23682 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:56.556572 23682 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.558447 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:18:56.559376 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92100.1us
I20260709 12:18:56.560209 19009 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.567754 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.527925 24071 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.569337 18442 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.569968 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.573434 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.571357 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.531875 23974 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.559265 23920 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:56.574859 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.532735 23873 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:18:56.572690 23883 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.573819 23931 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:18:56.577987 23821 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.577455 24016 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:18:56.585801 23921 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.627331 24073 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:18:56.627493 24072 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:56.619572 23931 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:18:56.628764 24077 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:18:56.615621 23944 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.780s	user 0.008s	sys 0.008s
I20260709 12:18:56.580152 24013 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:18:56.587767 24071 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:18:56.649605 18291 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:56.627396 24075 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:56.620129 23964 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.007s
W20260709 12:18:56.651840 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78475.1us
W20260709 12:18:56.652609 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90848.7us
I20260709 12:18:56.664877 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.665920 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:56.588302 23136 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.670403 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.671037 18370 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:56.671797 18375 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:56.672240 18372 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:18:56.652872 24020 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.000s
I20260709 12:18:56.655777 23931 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:56.668761 18368 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:56.668780 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:56.669709 18366 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:56.677493 18375 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:56.654157 23753 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.654997 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:56.672305 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.672822 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.655359 18442 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:56.678695 18370 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:18:56.679160 18447 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:56.678622 18367 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:56.681449 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.681985 18369 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:56.673465 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.673784 18372 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:56.662432 24068 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.657790 24072 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.675164 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:56.655019 23921 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.666937 23750 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:56.669104 23895 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.669709 24075 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:56.675591 23969 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:56.660483 23944 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.684710 24020 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:56.706845 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 119988us
I20260709 12:18:56.669184 23964 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.697870 24072 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:56.695509 23946 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.704493 24075 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.696158 24071 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:56.718797 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:56.719795 24080 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:56.718236 23936 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.752218 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.738348 23714 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.703934 23969 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:56.739696 18298 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:56.729492 23964 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.743673 18445 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:18:56.756606 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.758138 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:56.761031 24003 consensus_meta.cc:287] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
I20260709 12:18:56.762960 23944 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.759506 23969 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.763746 23942 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.763970 24025 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:56.764209 23942 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.764331 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.764616 23942 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.765759 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:56.766327 18443 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:56.767413 24088 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:18:56.767885 23985 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.793s	user 0.011s	sys 0.000s
I20260709 12:18:56.768074 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:56.768517 18440 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:18:56.769172 24088 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:56.769791 24088 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:56.770767 24050 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.764377 24025 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:56.772713 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
W20260709 12:18:56.765713 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.773487 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.783789 18367 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:56.772892 23714 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:56.777051 24088 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:56.791812 24007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.000s	sys 0.011s
W20260709 12:18:56.793205 24007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.000s	sys 0.011s
W20260709 12:18:56.797387 23986 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.000s
I20260709 12:18:56.797858 23986 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.798053 23931 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.781418 23873 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.790462 23714 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.792510 24088 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.780861 24075 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.800921 23944 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.805936 23816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.012s	sys 0.014s
I20260709 12:18:56.818200 19043 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:56.819996 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.804937 23714 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:56.808876 24083 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:18:56.824100 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.806480 24088 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.824414 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.826445 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.827069 19037 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:18:56.828105 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.825598 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.830120 18367 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:56.830953 24010 raft_consensus.cc:3055] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.830143 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.831526 24010 raft_consensus.cc:740] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:56.831872 19044 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:56.832417 24010 consensus_queue.cc:260] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.833473 24010 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.834393 23959 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.840178 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.840795 19040 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:56.841629 24013 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:56.841686 23921 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.842000 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.842186 24013 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:56.823444 24071 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:56.842866 24071 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.843339 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.844226 23921 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:56.843075 24013 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.846101 24013 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.816915 23944 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.803102 23873 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.872046 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.873011 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:56.874210 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.800273 24025 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:56.877310 18442 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:18:56.877421 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:56.817830 23816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.013s	sys 0.014s
I20260709 12:18:56.879823 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.880252 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:56.887161 23897 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:56.888659 23895 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.888857 24075 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.889233 23895 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.890448 23931 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.891314 23895 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:56.892335 23873 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.893308 24071 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.876139 23982 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.893854 23969 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.810981 24072 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.894654 23883 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.895079 19006 connection.cc:256] RPC call timeout handler was delayed by -1.1432907869998417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:18:56.898182 24095 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:18:56.880851 23820 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.878770 24068 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:56.905634 24072 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.902107 24011 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:56.906656 24011 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:56.907078 24011 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.910053 23714 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.912961 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.886767 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.915467 18442 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:56.917385 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.917778 18442 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:18:56.918696 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:56.919370 19043 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Advancing to term 12
I20260709 12:18:56.920706 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:56.922027 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.922699 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:56.924417 18447 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:18:56.924849 18447 pending_rounds.cc:85] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 11
I20260709 12:18:56.925163 18447 pending_rounds.cc:107] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:18:56.925411 18447 raft_consensus.cc:2889] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:56.925241 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.927011 19037 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:56.928023 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:56.929471 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:56.878628 23921 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:18:56.930552 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:56.899767 23946 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.929970 19037 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:18:56.894659 24089 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:56.934754 24082 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:56.924816 24080 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:18:56.936859 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:56.938284 24096 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.824980 24007 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:18:56.940160 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.924996 23986 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:56.942526 19038 raft_consensus.cc:606] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:56.979405 19038 raft_consensus.cc:614] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:56.901875 24020 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:56.931699 24071 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.980043 24020 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:56.961869 23884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.035s	sys 0.019s
I20260709 12:18:56.918561 24025 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.967633 19042 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:56.980449 24020 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:56.940723 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:56.933445 23921 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.943503 23750 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.977651 19043 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:56.881508 24003 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:18:56.903367 23883 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:56.932919 23999 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.939304 23897 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:18:56.985752 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:57.001209 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.943006 24068 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:18:56.989315 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:56.932370 23895 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:18:56.939481 23985 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:56.924405 23820 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:18:56.968999 24089 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:56.976438 24082 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:57.026393 24082 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.023375 23931 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:18:56.997416 23753 consensus_queue.cc:1059] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:56.996297 24085 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:57.042342 24085 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:57.042824 24085 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:57.046299 24011 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.047863 24102 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:18:57.048770 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.049345 18372 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:57.054662 19042 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:18:56.999871 23986 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:18:57.055941 23986 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:57.056385 23986 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.057405 23986 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:57.058403 23986 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.007752 24091 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:56.984251 23921 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:18:56.986651 23884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.035s	sys 0.020s
I20260709 12:18:57.025314 24089 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.025759 23714 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:57.055948 24088 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.060724 18372 raft_consensus.cc:2468] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:18:56.963708 23946 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.062356 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 83158.7us
W20260709 12:18:57.095757 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.106987 18366 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:57.107968 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.109666 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.109887 23969 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.110270 18375 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:57.110744 23946 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:57.103066 24091 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.117162 24088 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.117435 24091 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.119508 19040 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:57.120554 18375 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:18:57.104276 19009 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.135876 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.136710 18368 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:57.139117 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.140276 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.140599 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.140903 18375 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:18:57.146492 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 183812us
I20260709 12:18:57.054689 24020 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.124904 23883 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.123802 23944 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.156512 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.157509 19043 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:57.160717 19043 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
W20260709 12:18:57.146159 24006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.015s	sys 0.032s
I20260709 12:18:57.147058 24088 raft_consensus.cc:2804] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:57.162457 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.163220 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.118287 23942 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.146152 23996 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:57.168421 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.175495 19043 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:57.179693 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.181571 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.183853 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.184136 19035 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:57.185277 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.185827 19035 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:18:57.186592 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.184150 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.188452 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.189428 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.190109 19039 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:57.191195 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.196869 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:57.162084 24006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.016s	sys 0.032s
I20260709 12:18:57.199477 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:57.200249 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:57.199384 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.223321 19037 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:57.228020 19044 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:57.229741 19035 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:18:57.231910 19039 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:18:57.245677 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.233671 24019 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:57.254302 23753 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:57.279646 23750 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:57.280354 24052 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.356s	user 0.006s	sys 0.000s
I20260709 12:18:57.281272 23822 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:57.282157 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.283012 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2621) took 1626 ms. Trace:
I20260709 12:18:57.283424 18300 rpcz_store.cc:276] 0709 12:18:55.657038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.657156 (+   118us) service_pool.cc:224] Handling call
0709 12:18:55.665637 (+  8481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.666098 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:55.743532 (+ 77434us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.948719 (+205187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.949134 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.949264 (+   130us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:55.949296 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.949356 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.272164 (+1322808us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:18:57.282706 (+ 10542us) raft_consensus.cc:1685] finished
0709 12:18:57.282815 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.282946 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61207,"prepare.queue_time_us":106392,"prepare.run_cpu_time_us":1498,"prepare.run_wall_time_us":7805,"raft.queue_time_us":71831,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":170,"spinlock_wait_cycles":14232320,"thread_start_us":183299,"threads_started":2,"wal-append.queue_time_us":145077}
I20260709 12:18:57.285051 19041 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:57.285812 24085 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.288218 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.269593 24096 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:57.302165 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.300091 18444 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:57.303611 23883 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:57.304044 23883 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.304410 23883 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.309103 23921 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.311870 24020 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.273883 24011 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.260103 24088 raft_consensus.cc:697] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:18:57.319403 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 130100us
I20260709 12:18:57.306444 24082 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:57.321262 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.291929 23753 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.335801 24082 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:57.336897 24107 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:18:57.337111 23753 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.337292 24085 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.320604 18375 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:18:57.330401 24059 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.086s	user 0.004s	sys 0.000s
W20260709 12:18:57.341588 24010 consensus_meta.cc:287] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.000s
I20260709 12:18:57.330581 18368 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19. Ops: []
I20260709 12:18:57.342859 23942 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.343520 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.344146 23920 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:57.344553 23920 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:57.344684 24080 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:18:57.345450 23920 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.346107 24080 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:18:57.346365 24080 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:57.346417 23920 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.347088 24080 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.348446 24080 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:57.348507 23960 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:57.348954 23960 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.349269 24089 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.349741 23960 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.351749 24010 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:57.352828 24010 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.353789 23883 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.357311 18368 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:18:57.361819 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.350988 23960 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.366552 24018 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:57.278657 24086 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 12:18:57.368798 24125 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:57.370246 23883 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:18:57.395761 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.395903 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 824531us
I20260709 12:18:57.376870 24004 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:18:57.378611 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 854273us
I20260709 12:18:57.379637 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.399273 19041 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:57.379753 18263 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.527s (SENT) [suppressed 20 similar messages]
I20260709 12:18:57.380414 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:57.390579 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 209549us
I20260709 12:18:57.402984 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:57.403146 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.391291 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.404587 18368 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.406458 19035 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:57.373980 24010 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:57.411628 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.414112 19039 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.410729 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.427788 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:57.427887 18295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49948
I20260709 12:18:57.428556 18295 raft_consensus.cc:606] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Received request to transfer leadership
I20260709 12:18:57.440843 24105 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:57.441799 24088 consensus_queue.cc:237] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.444389 24115 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:18:57.447413 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49903.3us
I20260709 12:18:57.448554 23682 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.449071 18264 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:57.453100 23974 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.453572 23974 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:57.453924 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 55057us
I20260709 12:18:57.454452 23974 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.454872 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:57.404457 23896 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:57.457039 23519 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:57.457614 24050 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.392897 23919 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.422535 23921 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.456423 24017 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:18:57.458056 23821 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:18:57.457614 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:57.470175 18449 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:57.471490 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.471951 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:57.463222 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:57.478788 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.468297 24013 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.487771 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:57.487160 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.489259 18446 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:57.489200 19042 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:57.471100 23996 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:57.493747 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.472757 24089 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:57.417042 24004 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.497457 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.505318 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.505923 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:57.508280 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:57.509130 19035 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:57.509865 18368 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:57.496228 23964 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:57.521349 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.499424 23928 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:57.474364 23938 consensus_queue.cc:1059] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:18:57.500531 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.529944 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.531203 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.531798 19044 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:57.519608 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.501284 19041 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:57.533399 19039 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:57.513722 23936 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:18:57.514179 23896 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:18:57.513864 23894 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.517082 23722 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:57.495384 23968 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:18:57.480134 24050 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.533010 23964 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.514052 24017 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:57.539781 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.496592 24013 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.545161 23936 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.482795 23919 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.534808 24089 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:18:57.533313 23928 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.546263 23894 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:57.535277 24091 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:57.508394 23996 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:18:57.563702 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.565508 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.566490 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.555320 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.575987 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.616819 19042 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.549816 23821 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:57.550523 24085 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.550978 23964 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:57.621728 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 128058us
I20260709 12:18:57.623800 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.618021 18369 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.539134 24010 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:18:57.628809 18373 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.628823 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.629835 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.630604 18368 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.622375 24089 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.546571 24003 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:18:57.569190 24011 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:57.577436 23928 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.638342 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:57.643554 19042 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:18:57.644856 19042 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:18:57.645370 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.626197 24091 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:57.648384 23985 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:57.648555 24010 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:57.648797 23985 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.649106 24010 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.649570 24010 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.649747 23985 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.636106 23896 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:57.651467 23722 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:57.651857 24003 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:57.652238 24003 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.641731 23821 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:18:57.653492 24003 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.654624 23996 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.655293 24017 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:57.655748 24017 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.656117 24017 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:18:57.658811 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.575461 23936 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.659361 24075 consensus_queue.cc:1175] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3da7ad8fa20a4591963b660e4c848e7f" responder_term: 12 status { last_received { term: 12 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 12 index: 11 } }
W20260709 12:18:57.660480 24130 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:18:57.642103 24011 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:57.661996 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.662336 24011 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:57.662788 18449 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:57.664182 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:57.665113 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.651022 23985 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.673777 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:57.672559 18368 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
W20260709 12:18:57.673770 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.679518 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.685181 18297 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:18:57.686170 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:57.686964 18297 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:18:57.680090 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 171875us
W20260709 12:18:57.668696 24126 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:18:57.654637 24003 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.636701 24085 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.777110 23919 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:57.778599 24085 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:57.744943 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55516.4us
I20260709 12:18:57.678776 23722 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:57.779492 23722 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:57.779956 23722 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.632866 24068 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:18:57.783689 24068 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:57.784049 24068 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:57.784936 24068 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.786178 24068 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:57.785864 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:57.790458 18297 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:18:57.792429 23897 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:57.792861 23897 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:57.689761 23970 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:57.678548 23896 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:57.755201 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:57.642272 23894 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:57.772707 19042 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.773514 24013 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:57.774523 23986 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.001s
I20260709 12:18:57.775678 24004 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:57.747539 23938 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:18:57.795095 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.798239 19044 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:18:57.798332 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.793790 23897 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.801509 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 145445us
I20260709 12:18:57.821869 18292 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.792443 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:57.836190 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:57.837158 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44784.3us
I20260709 12:18:57.838155 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.765576 23714 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:57.829864 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.835264 19042 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:57.866127 18297 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:18:57.867674 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:57.868348 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:57.790987 23928 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.871785 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.874233 24085 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:57.874643 24085 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:57.875012 24085 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:57.879122 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 94171.1us
I20260709 12:18:57.898375 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:57.900790 23919 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:57.901176 23919 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:57.901525 23919 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:57.902848 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.903438 18375 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:57.904208 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.904603 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:57.905390 24025 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:18:57.905682 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.800355 24089 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.906414 24025 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:57.906875 24025 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:57.907612 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.907909 24025 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.908731 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.909742 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.911108 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:57.913130 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.913584 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:57.914175 19041 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
W20260709 12:18:57.905685 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.915338 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43631.6us
I20260709 12:18:57.886073 24140 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:57.886551 23722 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.890208 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2632) took 2041 ms. Trace:
I20260709 12:18:57.925732 18375 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:57.926337 18298 rpcz_store.cc:276] 0709 12:18:55.848800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.848937 (+   137us) service_pool.cc:224] Handling call
0709 12:18:55.849194 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.758055 (+908861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.889951 (+1131896us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400002154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:57.890151 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047412992}
I20260709 12:18:57.898931 24007 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:18:57.899595 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:57.927325 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.927646 18368 raft_consensus.cc:2376] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 11. Re-sending same reply.
W20260709 12:18:57.881843 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86845.3us
I20260709 12:18:57.924607 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.753279 23541 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 12:18:57.880192 24142 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:18:57.882660 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:57.920337 18292 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:57.786316 23964 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.988099 19043 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:18:57.930745 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:57.931010 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.931986 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:57.929716 23974 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.986945 23872 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.932828 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2635) took 2083 ms. Trace:
I20260709 12:18:57.985603 23996 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:57.991376 18300 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:57.928157 18373 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:57.992057 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.933516 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:57.993566 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:57.995026 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.000425 18375 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:58.001441 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.002218 18298 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:18:58.003176 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.752152 24091 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.840194 24013 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.004967 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 48) took 1219 ms. Trace:
I20260709 12:18:57.996533 18299 rpcz_store.cc:276] 0709 12:18:55.849669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.850693 (+  1024us) service_pool.cc:224] Handling call
0709 12:18:55.851051 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:55.883069 (+ 32018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.883098 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:55.883622 (+   524us) log.cc:844] Serialized 52 byte log entry
0709 12:18:57.495668 (+1612046us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18003fad20. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:57.673086 (+177418us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:57.926079 (+252993us) log.cc:844] Serialized 14 byte log entry
0709 12:18:57.927053 (+   974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.927123 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.927160 (+    37us) raft_consensus.cc:1685] finished
0709 12:18:57.932630 (+  5470us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.932772 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16324,"prepare.queue_time_us":73151,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":48528,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":65983,"spinlock_wait_cycles":16008064,"thread_start_us":418782,"threads_started":2,"wal-append.queue_time_us":1855513}
I20260709 12:18:57.932446 24142 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.981076 23936 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.009677 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.037014 18368 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:58.004961 23974 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.010885 23928 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.010193 23872 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:57.984607 23960 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.001s
I20260709 12:18:57.836233 24010 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.025120 18369 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:58.011444 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.067544 18297 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:18:58.068499 18371 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2510) took 2183 ms (client timeout 1617 ms). Trace:
I20260709 12:18:58.068686 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.069451 18297 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.027030 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.028301 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:18:58.023334 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.988453 24074 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.031661 19038 rpcz_store.cc:276] 0709 12:18:56.772480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.772601 (+   121us) service_pool.cc:224] Handling call
0709 12:18:57.991947 (+1219346us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54008205d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:57.992051 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389152128}
I20260709 12:18:58.032188 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.074580 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.008689 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:57.925607 23986 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.931066 24075 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:57.929944 23519 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:57.826296 23896 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:57.938948 23897 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.912635 23714 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.018453 24050 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.073200 18375 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:58.020362 23722 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.018838 23964 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:58.069096 18371 rpcz_store.cc:269] 0709 12:18:55.870795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.876378 (+  5583us) service_pool.cc:224] Handling call
0709 12:18:58.054396 (+2178018us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54000f8a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:58.054529 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279170432}
I20260709 12:18:58.052584 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:58.081537 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.097927 18298 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:18:58.097995 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.082660 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.083871 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.084946 23974 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.085454 23519 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.087390 23919 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.087725 24074 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.107497 24050 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.111012 24050 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.111419 24050 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.112391 24074 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.089426 24010 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.014672 24085 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.089838 24017 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.036996 24142 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:58.091857 23872 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.093390 18369 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:18:58.098697 18299 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:58.099653 18368 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 12:18:58.101840 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49406us
I20260709 12:18:58.130753 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:58.132221 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:58.104058 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.133648 23541 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.135107 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.106089 18375 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:58.107939 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:58.013451 24011 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:58.143617 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2750) took 2239 ms (client timeout 1628 ms). Trace:
W20260709 12:18:58.144111 18440 rpcz_store.cc:269] 0709 12:18:55.903705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.977896 (+ 74191us) service_pool.cc:224] Handling call
0709 12:18:58.143446 (+2165550us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54002049d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:58.143551 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795259136}
I20260709 12:18:58.110026 18297 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:18:58.145578 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.103302 24075 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:58.146651 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.147375 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.148159 18291 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
W20260709 12:18:58.149318 18263 leader_election.cc:336] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 2.178s (SENT)
I20260709 12:18:58.152320 18293 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:58.158499 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.160038 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.164004 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.164685 18446 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:58.148864 23872 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:58.143388 23541 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.164742 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.168678 18297 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:18:58.166899 23872 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.173374 23872 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.171038 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.175141 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.177548 18299 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:58.178584 18441 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:58.179632 18291 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:58.167776 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.181164 18299 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:58.182365 18297 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:18:58.186321 18367 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
W20260709 12:18:58.200125 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 268224us
I20260709 12:18:58.201035 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:58.202605 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:58.202611 18291 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:58.188513 24013 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:18:58.214977 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.206115 24007 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:58.218756 24007 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.220603 18415 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:18:58.222415 23919 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.214203 24089 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:58.227793 23816 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:58.302881 19036 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:58.306097 18299 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:58.308612 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 81993.3us
W20260709 12:18:58.316205 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.234627 24013 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:58.299536 23872 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.232142 24158 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:58.205145 24075 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.321048 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:58.256121 24089 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:18:58.301744 24068 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.309141 24159 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.335970 23963 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.334303 24089 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:58.341593 24068 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:18:58.347479 24166 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:58.372545 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.359501 24161 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:58.360870 24162 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:18:58.320734 23816 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:58.333726 24158 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.363767 24085 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.342810 23714 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.394728 24013 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:58.386807 24156 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:18:58.366669 24159 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:58.401664 24160 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:58.402208 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2252) took 1905 ms. Trace:
I20260709 12:18:58.402577 18294 rpcz_store.cc:276] 0709 12:18:56.497129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.497263 (+   134us) service_pool.cc:224] Handling call
0709 12:18:56.497613 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.497926 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.497951 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.502266 (+  4315us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.229710 (+1727444us) spinlock_profiling.cc:243] Waited 968 us on lock 0x7b18003d8160. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:58.335132 (+105422us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:58.335173 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.335274 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:58.401948 (+ 66674us) raft_consensus.cc:1685] finished
0709 12:18:58.401967 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.402155 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8824,"raft.queue_time_us":2110,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":14360,"spinlock_wait_cycles":2226432,"thread_start_us":105229,"threads_started":1,"wal-append.queue_time_us":1897259}
I20260709 12:18:58.404770 18294 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:58.422116 18299 raft_consensus.cc:1240] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:58.376106 24011 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.341152 23872 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.379335 24158 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:58.424235 24161 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:58.420553 24017 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.438516 24085 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:58.376181 24166 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.432708 24025 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:18:58.457818 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 136844us
I20260709 12:18:58.446278 24159 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:58.475636 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.464042 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:18:58.448484 24161 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:58.489588 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:58.451644 24025 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:18:58.452350 23816 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.503405 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.504067 18441 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:58.505154 18441 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:58.453145 24085 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.519949 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:58.521551 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:58.524729 24173 raft_consensus.cc:2764] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:18:58.526932 24167 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:58.527545 24105 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.528398 24017 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 12:18:58.538508 24164 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:18:58.539691 18448 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:18:58.540071 18448 pending_rounds.cc:85] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:18:58.540297 18448 pending_rounds.cc:107] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:18:58.540547 18448 raft_consensus.cc:2889] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:58.529330 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:58.541574 23894 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:58.541952 23894 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:58.542316 23894 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.543841 24025 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.531546 24142 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.545107 24025 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.527913 23964 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:58.545317 18298 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:18:58.543838 23541 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:18:58.550261 24050 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.542804 23974 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:18:58.555795 23894 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.556788 23872 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.528928 24068 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:58.560110 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2517) took 1089 ms. Trace:
I20260709 12:18:58.628295 18445 rpcz_store.cc:276] 0709 12:18:57.470391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.489527 (+ 19136us) service_pool.cc:224] Handling call
0709 12:18:57.489889 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.522659 (+ 32770us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:57.616386 (+ 93727us) log.cc:844] Serialized 14 byte log entry
0709 12:18:57.913695 (+297309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.913763 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:57.913826 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.559812 (+645986us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.559885 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.560043 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72835,"prepare.queue_time_us":223628,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":4169,"raft.queue_time_us":73304,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7344,"spinlock_wait_cycles":74774912,"thread_start_us":302630,"threads_started":2,"wal-append.queue_time_us":443070}
I20260709 12:18:58.561771 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1280) took 1376 ms. Trace:
I20260709 12:18:58.630954 19036 rpcz_store.cc:276] 0709 12:18:57.185239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.234839 (+ 49600us) service_pool.cc:224] Handling call
0709 12:18:57.254717 (+ 19878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.307152 (+1052435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.561503 (+254351us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400603454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.561701 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409384320}
I20260709 12:18:58.563875 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:58.633188 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69386.1us
I20260709 12:18:58.565634 24085 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:58.566376 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.657800 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.568835 23896 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.578560 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.663264 19039 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEARNER]: Advancing to term 14
I20260709 12:18:58.665172 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1300) took 1352 ms. Trace:
I20260709 12:18:58.665716 18372 rpcz_store.cc:276] 0709 12:18:57.312965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.313220 (+   255us) service_pool.cc:224] Handling call
0709 12:18:57.313547 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.336821 (+ 23274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.336852 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.336896 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:18:57.336930 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.664874 (+1327944us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b5400414e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.664928 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.665104 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53090560}
I20260709 12:18:58.580945 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:18:58.668009 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87173.6us
I20260709 12:18:58.668915 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:58.671758 23962 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:58.672286 23962 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.582103 18299 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:58.679483 23722 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:58.680261 23722 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.680444 23136 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:58.681066 23136 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:58.681569 23136 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.582808 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:58.684250 18296 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.583633 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:58.688218 19037 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:18:58.689353 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.690521 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:58.585712 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:58.589008 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:58.591952 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:58.594872 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.692772 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.693392 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.693799 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.694487 24084 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.596726 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:58.695608 19035 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:18:58.696725 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:58.695143 24084 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.698055 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.699419 24068 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.699491 24084 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:58.700079 24068 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.702348 24068 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:58.597539 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.598502 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:58.703540 19041 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:58.704458 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.603693 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:58.706151 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 102568us
I20260709 12:18:58.706965 18263 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:58.607741 19042 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:18:58.507838 24161 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:18:58.708341 23936 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.709028 23936 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.709537 19042 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:18:58.709488 23936 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.612643 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49948 (request call id 0) took 1177 ms. Trace:
I20260709 12:18:58.617506 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:58.610661 23821 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:58.661273 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:58.662027 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.681480 23722 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.673391 23962 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:58.741731 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.713745 24017 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:18:58.718776 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:58.759536 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.703080 19043 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:58.760536 18366 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:58.760721 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:58.714272 23541 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:58.762094 23541 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.691039 19037 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:58.763252 18375 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:58.764364 19037 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:58.766237 18375 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:58.767053 19043 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:18:58.767321 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:58.768790 19041 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:18:58.769378 23964 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:58.768870 23541 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.769775 23964 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.771377 23962 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:58.738616 18266 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:58.714938 23936 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.490821 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:58.741273 18372 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:58.726642 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.775635 18442 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:58.777724 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:58.711184 19042 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:58.699157 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:58.739786 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:58.753733 23821 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.757782 18368 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:58.757917 18196 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:58.759663 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:58.751992 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.781457 18374 raft_consensus.cc:2410] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:18:58.714584 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.781601 18196 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 9 to 12, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 12 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:58.770825 23964 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.784461 23964 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:58.784296 18196 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 12 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 13 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:58.782974 23821 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.786656 23821 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:58.772953 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 102713us
I20260709 12:18:58.757357 18295 rpcz_store.cc:276] 0709 12:18:57.427611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.427765 (+   154us) service_pool.cc:224] Handling call
0709 12:18:58.604383 (+1176618us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b5400473720. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df9520 00007f3c68e76652 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.604700 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92434176}
I20260709 12:18:58.787720 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:58.735121 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:58.789788 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54788.6us
I20260709 12:18:58.791431 24068 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:58.735340 23884 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:18:58.792490 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:58.792608 23884 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:58.793128 23884 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:58.793357 23968 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.793788 23968 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.794139 23968 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.795646 18295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49948
I20260709 12:18:58.798974 18374 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:58.801493 18296 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:58.802057 23968 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.802639 23964 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:58.567137 24158 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.804812 23974 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:18:58.807480 23928 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:58.838436 24017 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.809337 23938 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:18:58.811654 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.787665 18196 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:58.827082 24013 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:58.815565 23136 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.827732 24121 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.191s
I20260709 12:18:58.818449 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:18:58.867736 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49381.2us
I20260709 12:18:58.825722 19039 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEARNER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:58.826333 23936 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:58.622790 23999 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.827452 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 969) took 1913 ms. Trace:
I20260709 12:18:58.892486 23999 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.828473 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2637) took 1156 ms. Trace:
I20260709 12:18:58.893615 19038 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:58.803785 24138 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:58.893877 23873 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:58.627554 23519 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.894484 23873 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:58.894811 23999 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.895495 24174 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.895452 23873 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.568869 24089 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:58.896131 24174 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.898051 24174 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.841475 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:58.840816 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:18:58.840974 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 223579us
I20260709 12:18:58.901180 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:58.657480 23883 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:58.846673 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:58.794145 23884 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.847533 24114 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:18:58.819461 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:58.875866 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.911618 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:58.914726 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:58.914809 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:58.915940 18441 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:58.908737 18374 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:18:58.875808 18196 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 10 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:18:58.818154 18192 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 296) took 17328 ms (client timeout 15000 ms). Trace:
I20260709 12:18:58.890569 23894 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.865657 23938 consensus_queue.cc:1243] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:18:58.891497 24166 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:58.892422 19040 rpcz_store.cc:276] 0709 12:18:56.913680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.913836 (+   156us) service_pool.cc:224] Handling call
0709 12:18:56.914131 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.123850 (+209719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.827176 (+1703326us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400808254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.827384 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471811968}
W20260709 12:18:58.832479 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 246814us
I20260709 12:18:58.853561 24104 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:18:58.898101 24089 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:58.843544 18368 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:58.903064 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:58.903935 18371 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:58.904577 19036 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:58.905911 23136 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:58.859645 23896 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:18:58.923211 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.924460 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:58.904596 24011 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.893328 18293 rpcz_store.cc:276] 0709 12:18:57.671648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.672041 (+   393us) service_pool.cc:224] Handling call
0709 12:18:57.753218 (+ 81177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.152733 (+399515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.828229 (+675496us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b540040e554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.828418 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917268352}
I20260709 12:18:58.940568 18446 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:18:58.931491 18196 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 9 to 15, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 15 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:58.932432 24084 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:58.933207 23896 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.864966 24013 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.943476 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:58.990893 23136 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:18:58.991806 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 67443.5us
I20260709 12:18:58.993673 19006 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:58.991554 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48172.4us
I20260709 12:18:58.995163 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:58.900333 23936 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:58.938571 24075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.045s	sys 0.109s
I20260709 12:18:59.006762 18440 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:59.006791 23136 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:58.934015 24088 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:58.930039 23872 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:58.918843 24074 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:18:59.032846 18367 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.031263 23136 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:58.998190 24091 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:18:59.018347 24075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.045s	sys 0.109s
I20260709 12:18:58.934288 23883 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:59.004895 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:59.008737 24114 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:58.998679 24121 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:58.998517 24089 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:59.029963 23946 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:59.059562 18444 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:59.060346 18367 raft_consensus.cc:2410] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:18:59.040710 24093 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.990231 24011 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.046047 24091 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:18:59.046479 23883 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.924129 23959 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 12:18:59.080952 18196 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 12 to 15, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:18:59.077244 24093 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.080952 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:18:59.083809 18445 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.066586 23946 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.041501 23872 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:59.081895 24076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.022s	sys 0.026s
W20260709 12:18:59.130292 24076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.023s	sys 0.027s
I20260709 12:18:59.131343 23936 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:59.133342 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:59.133176 18196 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 7 to 10, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 10 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:59.134430 18448 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:18:59.149390 18196 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:59.158245 24007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.022s	sys 0.035s
I20260709 12:18:59.081789 23968 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:59.069514 24089 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.143150 23946 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:59.188279 19044 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.022s
I20260709 12:18:59.182271 18295 raft_consensus.cc:606] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Received request to transfer leadership
W20260709 12:18:59.077056 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.016s	sys 0.035s
W20260709 12:18:59.176072 18370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.021s	sys 0.017s
I20260709 12:18:59.084934 23883 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.189950 18295 raft_consensus.cc:614] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:59.178946 18445 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:18:59.190317 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.017s	sys 0.035s
W20260709 12:18:59.190635 18370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.021s	sys 0.018s
W20260709 12:18:59.175972 24007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.022s	sys 0.035s
I20260709 12:18:59.174048 24158 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:18:59.189844 19044 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.013s	sys 0.022s
W20260709 12:18:59.191294 24010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.012s	sys 0.021s
W20260709 12:18:59.182431 24096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.010s	sys 0.023s
W20260709 12:18:59.231297 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 767300us
I20260709 12:18:59.140894 24121 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:18:59.235972 18264 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:59.187239 24050 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.176386 24076 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:59.175200 24163 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:59.248047 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:18:59.177084 23714 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:18:59.186918 24085 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:18:59.226332 18443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.009s	sys 0.019s
I20260709 12:18:59.258905 18296 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:18:59.259407 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.306384 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:18:59.259553 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.307535 18298 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
W20260709 12:18:59.236094 24026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.009s	sys 0.008s
I20260709 12:18:59.230981 24114 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.234825 24159 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.085119 23896 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.275207 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:59.275615 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.311264 18299 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
W20260709 12:18:59.254552 24096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.012s	sys 0.024s
W20260709 12:18:59.256598 24025 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.709s	user 0.008s	sys 0.004s
I20260709 12:18:59.278764 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.292873 23722 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:59.282573 18449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.012s	sys 0.020s
W20260709 12:18:59.280938 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 438656us
W20260709 12:18:59.236850 24010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.014s	sys 0.022s
I20260709 12:18:59.285508 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:18:59.321673 18449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.013s	sys 0.020s
I20260709 12:18:59.296380 18445 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:59.255854 24158 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:59.304651 24007 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.302259 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54264.2us
I20260709 12:18:59.364570 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:18:59.308640 18443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.010s	sys 0.020s
W20260709 12:18:59.282658 24161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.022s	sys 0.032s
I20260709 12:18:59.313632 18293 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:18:59.379494 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 454093us
I20260709 12:18:59.382663 18292 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
W20260709 12:18:59.427166 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.314884 24026 raft_consensus.cc:476] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 12:18:59.299440 24013 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.277745 24076 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:59.302731 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.278764 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:59.431293 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66808.3us
W20260709 12:18:59.380559 23541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.010s	sys 0.015s
I20260709 12:18:59.323170 23896 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:18:59.430846 24178 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
W20260709 12:18:59.374760 24161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.022s	sys 0.032s
I20260709 12:18:59.355712 24096 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:59.435262 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.441461 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 59681.9us
W20260709 12:18:59.447991 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 169287us
I20260709 12:18:59.474059 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:59.310017 24085 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.306612 24163 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:59.456213 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.404760 23873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.020s	sys 0.011s
I20260709 12:18:59.313025 24114 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.369119 23722 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:59.405121 23963 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.474675 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:59.372572 24158 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.475740 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.476306 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.459255 24011 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.480139 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:18:59.367146 23938 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:18:59.447019 24075 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:18:59.455869 23541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.010s	sys 0.015s
I20260709 12:18:59.446789 24074 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:59.452517 24026 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:59.456789 23894 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:59.475435 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:59.497223 24010 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:18:59.497596 24182 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:59.497540 24085 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:59.490275 23928 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:59.498167 24089 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:59.494913 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:18:59.467819 24138 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.018s
I20260709 12:18:59.502192 18373 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:59.502533 18374 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:18:59.472745 23872 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.480993 24096 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.505641 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:59.494505 24184 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:18:59.528621 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:18:59.490674 23873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.020s	sys 0.011s
I20260709 12:18:59.520112 23928 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:59.507076 24075 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:59.508203 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.471961 24013 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:18:59.522996 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:18:59.545236 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.517777 23938 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:18:59.547483 23872 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:59.548015 23872 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:18:58.922619 18192 rpcz_store.cc:269] 0709 12:18:41.490094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.490369 (+   275us) service_pool.cc:224] Handling call
0709 12:18:42.445358 (+954989us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:18:58.818057 (+16372699us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:53.326794 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:53.327646 (+   852us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:53.327708 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:18:53.327735 (+    27us) tablet.cc:662] Decoding operations
0709 12:18:53.399364 (+ 71629us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:53.399469 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:53.399495 (+    26us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:18:53.633159 (+233664us) tablet.cc:701] Row locks acquired
0709 12:18:53.648996 (+ 15837us) write_op.cc:260] PREPARE: finished
0709 12:18:53.694824 (+ 45828us) write_op.cc:270] Start()
0709 12:18:53.747936 (+ 53112us) write_op.cc:276] Timestamp: P: 1783599533658649 usec, L: 0
0709 12:18:53.762093 (+ 14157us) op_driver.cc:348] REPLICATION: starting
0709 12:18:53.989988 (+227895us) log.cc:844] Serialized 10735 byte log entry
0709 12:18:55.046355 (+1056367us) op_driver.cc:464] REPLICATION: finished
0709 12:18:55.319915 (+273560us) write_op.cc:301] APPLY: starting
0709 12:18:55.320030 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:55.550932 (+230902us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:55.551084 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:55.828867 (+277783us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:55.829287 (+   420us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 12:18:55.829333 (+    46us) write_op.cc:312] APPLY: finished
0709 12:18:55.829883 (+   550us) log.cc:844] Serialized 236 byte log entry
0709 12:18:55.861548 (+ 31665us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:55.861837 (+   289us) write_op.cc:454] Released schema lock
0709 12:18:55.861972 (+   135us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:55.862576 (+   604us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:55.862619 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":342915,"reported_tablets":42,"spinlock_wait_cycles":344862592,"child_traces":[["op",{"apply.queue_time_us":272416,"apply.run_cpu_time_us":50888,"apply.run_wall_time_us":542614,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116879,"num_ops":27,"prepare.queue_time_us":72673,"prepare.run_cpu_time_us":24951,"prepare.run_wall_time_us":895600,"raft.queue_time_us":370517,"raft.run_cpu_time_us":33550,"raft.run_wall_time_us":198838,"replication_time_us":1253917,"thread_start_us":439469,"threads_started":5,"wal-append.queue_time_us":528000,"wal-append.run_cpu_time_us":13023,"wal-append.run_wall_time_us":2327905}]]}
I20260709 12:18:59.548933 23872 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.550964 24075 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.526021 24011 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:18:59.551944 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:59.552356 24011 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:59.552889 24011 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:59.550899 24089 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:18:59.554631 24089 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:59.554952 24089 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:18:59.555807 24089 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.557036 24089 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.556978 23722 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.553051 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:59.533269 23928 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:59.570700 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 499518us
I20260709 12:18:59.507274 23962 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:18:59.494850 24161 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
W20260709 12:18:59.523921 24138 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.009s	sys 0.018s
I20260709 12:18:59.514997 24074 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.605504 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.606168 18373 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:18:59.531526 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
W20260709 12:18:59.537338 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.607820 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.504172 23963 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:59.582954 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52668.6us
W20260709 12:18:59.587139 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44786.2us
I20260709 12:18:59.527813 24096 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.493013 24025 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:18:59.378417 24007 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.596931 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.616729 18369 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:18:59.514919 24010 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.584442 23816 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:59.584980 23873 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:18:59.509624 23936 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:18:59.609812 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:59.612638 18447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.008s	sys 0.047s
W20260709 12:18:59.602360 24187 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.112s	user 0.002s	sys 0.001s
I20260709 12:18:59.586613 24196 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:59.614434 18373 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:18:59.614938 18266 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:59.672354 18266 leader_election.cc:403] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:18:59.672756 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:59.584686 23541 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:59.540925 23968 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:59.621050 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.665966 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.604561 24075 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:59.666855 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:59.667833 18447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.008s	sys 0.048s
I20260709 12:18:59.623286 24161 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.670738 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:18:59.672811 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60818us
I20260709 12:18:59.710875 18366 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.528870 24121 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:18:59.673804 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2540) took 1007 ms. Trace:
I20260709 12:18:59.673583 18292 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.617959 24010 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:59.614023 23962 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 12:18:59.684973 24185 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:18:59.686421 18373 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:18:59.687062 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:18:59.671774 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:18:59.734115 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.705374 24193 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:18:59.713239 18367 rpcz_store.cc:276] 0709 12:18:58.659674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.661693 (+  2019us) service_pool.cc:224] Handling call
0709 12:18:59.667051 (+1005358us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54002e7a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:59.667257 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295487872}
W20260709 12:18:59.734642 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.709434 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:59.668489 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:18:59.735584 18368 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:59.723701 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.731321 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.684859 24138 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:18:59.680770 24114 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:59.671736 24196 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:18:59.641228 24194 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
W20260709 12:18:59.714795 24181 log.cc:927] Time spent T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:18:59.645812 23816 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.768492 18367 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:18:59.645493 23873 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.741536 19036 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:18:59.744776 18366 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:59.715464 18299 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:59.749893 18292 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:59.753077 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.679750 24026 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.783556 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2531) took 1070 ms. Trace:
I20260709 12:18:59.783423 24026 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.732072 23541 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:59.783982 18443 rpcz_store.cc:276] 0709 12:18:58.713548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.714735 (+  1187us) service_pool.cc:224] Handling call
0709 12:18:58.715041 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.715424 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.715451 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.715996 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.365588 (+649592us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:59.365646 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.365747 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.783359 (+417612us) raft_consensus.cc:1685] finished
0709 12:18:59.783380 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.783501 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3633,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"thread_start_us":649174,"threads_started":1,"wal-append.queue_time_us":745962}
I20260709 12:18:59.732014 23968 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:59.762457 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.788163 18296 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:59.702955 24076 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:18:59.727497 24179 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:18:59.715286 24201 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.767644 24085 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.767709 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 400655us
I20260709 12:18:59.778379 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.779258 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.815563 19043 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:18:59.781890 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.816613 19036 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:18:59.817700 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:18:59.818274 19036 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:59.786011 18293 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:59.755380 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:18:59.819976 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.740671 18374 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:18:59.786931 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:18:59.799769 18297 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:18:59.820973 18297 pending_rounds.cc:85] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 12
I20260709 12:18:59.821202 18297 pending_rounds.cc:107] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:18:59.821455 18297 raft_consensus.cc:2889] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.800347 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.764118 24200 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.819029 18293 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:18:59.800770 24188 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:18:59.771572 24114 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.776543 24158 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.844019 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 985) took 1252 ms. Trace:
I20260709 12:18:59.777973 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.844388 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.823709 19035 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:18:59.820653 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 249207us
I20260709 12:18:59.844406 19038 rpcz_store.cc:276] 0709 12:18:58.591442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.591623 (+   181us) service_pool.cc:224] Handling call
0709 12:18:59.843885 (+1252262us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400693054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:18:59.843971 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462611328}
I20260709 12:18:59.801263 18368 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:18:59.819329 18373 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:18:59.746951 24161 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.840950 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.869472 18294 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:18:59.825402 18293 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.870237 18293 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.788003 24198 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:59.880363 19035 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:18:59.881567 19036 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
W20260709 12:18:59.845243 24088 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.808s	user 0.010s	sys 0.002s
I20260709 12:18:59.820423 24026 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:18:59.889003 18367 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:18:59.889294 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.841542 24203 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.894932 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1290) took 1286 ms. Trace:
I20260709 12:18:59.896517 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2274) took 2790 ms. Trace:
I20260709 12:18:59.899276 18370 rpcz_store.cc:276] 0709 12:18:57.105790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.105947 (+   157us) service_pool.cc:224] Handling call
0709 12:18:57.106345 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:57.106705 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.106729 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:57.107160 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.499180 (+1392020us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b1800404740. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:18:59.231151 (+731971us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:59.231187 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.231268 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.877719 (+646451us) raft_consensus.cc:1685] finished
0709 12:18:59.877745 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.896443 (+ 18698us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2134,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8250,"spinlock_wait_cycles":11985152,"thread_start_us":731768,"threads_started":1,"wal-append.queue_time_us":2203175}
W20260709 12:18:59.905450 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 285258us
I20260709 12:18:59.806367 23968 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.902382 19044 rpcz_store.cc:276] 0709 12:18:58.608457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.608692 (+   235us) service_pool.cc:224] Handling call
0709 12:18:58.608991 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.609238 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:59.233618 (+624380us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.894453 (+660835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.894503 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:59.894541 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.894773 (+   232us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.894872 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":833956,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":5606,"thread_start_us":1279186,"threads_started":3,"wal-append.queue_time_us":551913}
I20260709 12:18:59.910861 19036 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEARNER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.911963 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2532) took 1198 ms. Trace:
I20260709 12:18:59.806515 23541 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:18:59.887181 24191 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
W20260709 12:18:59.939036 24199 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:18:59.825498 24085 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:18:59.954386 18293 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:18:59.954866 18293 pending_rounds.cc:85] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 13
I20260709 12:18:59.912745 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:18:59.879801 24205 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:18:59.938207 18449 rpcz_store.cc:276] 0709 12:18:58.713853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.716564 (+  2711us) service_pool.cc:224] Handling call
0709 12:18:58.716998 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.717399 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.717439 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.717972 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.322057 (+604085us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:59.322094 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.322166 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.911755 (+589589us) raft_consensus.cc:1685] finished
0709 12:18:59.911775 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.911892 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1957,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":603763,"threads_started":1,"wal-append.queue_time_us":775938}
I20260709 12:18:59.818883 24201 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:18:59.914126 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 211258us
W20260709 12:18:59.953380 24189 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:18:59.951265 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:18:59.991956 18294 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:18:59.951421 24142 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:59.826967 24196 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.952085 18292 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:18:59.951977 23963 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:18:59.867561 24114 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:00.028657 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 183047us
I20260709 12:18:59.840801 24200 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.964654 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:18:59.965888 18373 raft_consensus.cc:1240] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:18:59.932989 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:18:59.909508 24203 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:18:59.980950 18293 pending_rounds.cc:107] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 12:19:00.054366 18293 raft_consensus.cc:2889] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.938740 24026 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:18:59.937906 24074 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:18:59.966358 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:00.077564 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 111290us
I20260709 12:19:00.078639 18266 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:59.993237 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.080756 19044 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:19:00.081152 23946 consensus_meta.cc:287] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.000s
I20260709 12:19:00.020151 24194 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.894846 23999 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:59.999701 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 230442us
I20260709 12:19:00.084105 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:00.002681 18263 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Timed out: RequestConsensusVote RPC to 127.16.193.68:43821 timed out after 1.527s (SENT)
I20260709 12:19:00.086032 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.004539 23896 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.880053 24158 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.977017 24195 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:18:59.887537 24198 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.038889 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:18:59.962214 24138 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.038555 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:00.044903 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 112049us
I20260709 12:18:59.907104 24088 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:59.997920 24007 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:00.082496 23974 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:19:00.108712 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.086917 24190 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:19:00.082041 24217 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.083971 24076 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.991179 24205 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.123353 24103 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:00.124089 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 85273.9us
I20260709 12:19:00.125888 24202 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.126387 24076 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.129897 24147 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:00.136054 24177 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.854s	user 0.001s	sys 0.005s
I20260709 12:19:00.118065 24138 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.136579 24203 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.137964 24114 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.140560 23968 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.093576 23962 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.095484 24194 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.143075 24233 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:00.136086 24198 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.149463 24093 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.153282 24180 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:00.088918 24074 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.089416 23722 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:00.096705 24142 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.081765 23946 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.085916 24026 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.137652 24197 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:00.122161 23816 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.095000 23896 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.095240 24158 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.170941 24202 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:00.182428 24011 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.002s
I20260709 12:19:00.191262 24180 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:00.187762 24114 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.211000 24200 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.210883 24114 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.212073 24180 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.212698 24194 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.213122 24180 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.213670 24114 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.215598 24201 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.216507 24085 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.216861 24091 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.171741 24103 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.218624 24091 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:00.208371 24096 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 12:19:00.219668 24103 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.211529 24076 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.217684 24158 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.219419 24096 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.159643 24217 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.193157 24161 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.207141 24202 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.198205 24197 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.206614 24233 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:00.180012 24147 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.230695 24197 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.231084 24205 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.226823 24233 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:00.232039 24197 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:00.232689 24233 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.231787 24217 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:00.092625 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 90003.6us
W20260709 12:19:00.268991 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.270992 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.268939 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.275385 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.280474 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1092) took 1022 ms. Trace:
I20260709 12:19:00.281232 18291 rpcz_store.cc:276] 0709 12:18:59.257813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.259285 (+  1472us) service_pool.cc:224] Handling call
0709 12:18:59.282985 (+ 23700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.283224 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:59.787147 (+503923us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.278863 (+491716us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.279299 (+   436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.279410 (+   111us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:00.279552 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.279753 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.280134 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":622515,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2550,"thread_start_us":969847,"threads_started":2,"wal-append.queue_time_us":444138}
I20260709 12:19:00.284472 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.287170 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.290515 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.292201 18291 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:00.293569 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.294677 18291 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:00.295961 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:00.297024 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.298962 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.299813 18294 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:00.300822 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:00.315881 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.318081 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.003s	sys 0.003s
W20260709 12:19:00.318279 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 232555us
W20260709 12:19:00.318853 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.003s	sys 0.004s
W20260709 12:19:00.320575 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.321727 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.151921 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.356139 23722 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.172967 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.360090 19038 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:19:00.361972 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1365) took 1719 ms. Trace:
I20260709 12:19:00.184393 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.362517 18447 rpcz_store.cc:276] 0709 12:18:58.642156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.642324 (+   168us) service_pool.cc:224] Handling call
0709 12:18:58.642701 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.642978 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:58.838776 (+195798us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.960178 (+1121402us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.323966 (+363788us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.324085 (+   119us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:00.353660 (+ 29575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.361352 (+  7692us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.361643 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":629961,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":169018,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":1474521,"threads_started":3,"wal-append.queue_time_us":1004485}
I20260709 12:19:00.364770 19044 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:00.366048 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.366221 24200 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.144779 23541 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.371538 23541 consensus_queue.cc:1059] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:00.372776 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49668.2us
I20260709 12:19:00.200986 24080 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:00.323632 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.374051 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.367036 24200 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:00.367444 24121 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.013s	sys 0.010s
I20260709 12:19:00.369439 24138 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.370185 24217 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.372928 23541 consensus_queue.cc:1243] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:19:00.185611 23962 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:00.360105 19039 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:00.374802 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51889.6us
I20260709 12:19:00.366959 18447 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:00.367153 23896 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.389681 24198 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:00.391408 24205 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.393318 18369 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:00.394273 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:00.395398 24246 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:00.396432 24147 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.195940 24074 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.397665 23962 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.397756 24200 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.398581 24237 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:00.399781 24089 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.842s	user 0.005s	sys 0.015s
I20260709 12:19:00.203295 24010 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:00.383769 24121 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.013s	sys 0.010s
I20260709 12:19:00.374065 24080 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:00.404438 18291 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:00.467944 18291 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.397404 23919 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.404994 24011 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.393363 23946 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.407825 23968 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:00.390174 24217 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.415623 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.397372 24026 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.437803 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:00.434819 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49948 (request call id 1) took 1643 ms. Trace:
I20260709 12:19:00.489686 18295 rpcz_store.cc:276] 0709 12:18:58.791713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.795525 (+  3812us) service_pool.cc:224] Handling call
0709 12:19:00.434375 (+1638850us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b540040db54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:00.434579 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887738240}
W20260709 12:19:00.490144 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52572.9us
I20260709 12:19:00.491096 23873 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.492084 18295 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:19:00.492980 23897 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.493075 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.410946 24246 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:00.495575 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 86098.9us
I20260709 12:19:00.496024 24246 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:00.495800 18291 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.497751 18291 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.498454 19041 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:00.498826 19041 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:00.358701 23722 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.500519 24080 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.501273 18295 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:00.422778 24096 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.426643 24222 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.451929 24226 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:00.457190 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78296.8us
I20260709 12:19:00.460062 24245 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:00.453394 24237 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.464701 24010 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.476301 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1363) took 2375 ms. Trace:
I20260709 12:19:00.480515 18449 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:00.487143 18442 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:00.475035 24011 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.501997 24080 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.509747 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.510161 23873 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:00.512102 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.512898 24013 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:00.513628 24217 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.484998 24205 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.529276 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.530054 18441 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:00.530463 24163 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.531001 24205 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.531208 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.516065 24222 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.517163 24226 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:00.532291 18441 raft_consensus.cc:2410] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:19:00.532778 24196 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:00.518352 23884 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:00.520800 24075 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.005s
I20260709 12:19:00.507874 24237 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.508455 18444 rpcz_store.cc:276] 0709 12:18:58.101264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.101890 (+   626us) service_pool.cc:224] Handling call
0709 12:18:58.102245 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.067345 (+965100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.475748 (+1408403us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54001a6154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.476098 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201144064}
I20260709 12:19:00.523810 18448 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:00.537616 18368 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:00.535156 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.539204 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:00.539441 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.539487 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:00.536649 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:00.547914 18373 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:19:00.548159 18374 raft_consensus.cc:2410] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:00.549716 18374 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:00.524427 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.527809 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.555192 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2789) took 1008 ms. Trace:
I20260709 12:19:00.556021 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.557190 19036 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:00.558465 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.559878 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.561717 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.565573 24222 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.528244 23873 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.566330 23873 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.559110 19036 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:00.571523 24074 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.572489 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2788) took 1025 ms. Trace:
I20260709 12:19:00.572950 18442 rpcz_store.cc:276] 0709 12:18:59.546755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.552838 (+  6083us) service_pool.cc:224] Handling call
0709 12:19:00.572252 (+1019414us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b5400422f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:00.572424 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083811712}
I20260709 12:19:00.575701 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.532938 24226 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.528548 24013 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:00.555742 18441 rpcz_store.cc:276] 0709 12:18:59.547031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.533008 (+985977us) service_pool.cc:224] Handling call
0709 12:19:00.533884 (+   876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.554777 (+ 20893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.554812 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.554877 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:00.554918 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.554967 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.555109 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:00.582654 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:00.594498 19039 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:19:00.597231 23928 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 1.027s	user 0.015s	sys 0.013s
I20260709 12:19:00.622669 18291 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:00.622538 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.631171 18295 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:19:00.628692 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2796) took 1015 ms. Trace:
I20260709 12:19:00.631955 18442 rpcz_store.cc:276] 0709 12:18:59.609815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.574115 (+964300us) service_pool.cc:224] Handling call
0709 12:19:00.580499 (+  6384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.624313 (+ 43814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.624361 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.624412 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.624465 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.624556 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.624733 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:00.617569 19041 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.639209 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:00.645470 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.648422 18447 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:19:00.644474 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2793) took 1057 ms. Trace:
I20260709 12:19:00.617944 23928 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:00.656145 24249 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:19:00.658638 24159 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.348s	user 0.013s	sys 0.010s
I20260709 12:19:00.658531 23928 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:00.659768 23883 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.424s	user 0.021s	sys 0.001s
I20260709 12:19:00.656145 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.660333 23883 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.661144 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:00.663475 23964 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:19:00.663844 23883 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.664062 23936 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:00.664652 23936 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.665076 23936 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:00.665691 23964 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:00.663519 19042 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:00.667733 24013 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.668447 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.669602 24013 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:00.672432 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.673180 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2293) took 1176 ms. Trace:
I20260709 12:19:00.673570 18371 rpcz_store.cc:276] 0709 12:18:59.496308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.496461 (+   153us) service_pool.cc:224] Handling call
0709 12:18:59.498724 (+  2263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.507480 (+  8756us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:59.824278 (+316798us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.155925 (+331647us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b18000f3b80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:19:00.407169 (+251244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.407329 (+   160us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:00.407470 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.672764 (+265294us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b54004150d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.672810 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.673012 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":489339,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":87144,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"spinlock_wait_cycles":83113088,"thread_start_us":742734,"threads_started":3,"wal-append.queue_time_us":384591}
I20260709 12:19:00.659705 18300 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:00.677158 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.678630 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.681007 24011 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.681489 24011 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:00.675799 24245 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.681878 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2790) took 1134 ms. Trace:
W20260709 12:19:00.682173 24244 log.cc:927] Time spent T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.450s	user 0.000s	sys 0.007s
I20260709 12:19:00.682391 18449 rpcz_store.cc:276] 0709 12:18:59.559338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.547296 (+987958us) service_pool.cc:224] Handling call
0709 12:19:00.625687 (+ 78391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.681423 (+ 55736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.681470 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.681548 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.681594 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.681656 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.681796 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:00.682416 24245 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.684991 23519 raft_consensus.cc:3055] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:00.685278 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87464.8us
I20260709 12:19:00.685454 24245 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.669615 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:00.693706 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.689778 18298 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:19:00.675906 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.695156 19037 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:00.696223 24254 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:19:00.686772 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2794) took 1083 ms. Trace:
I20260709 12:19:00.685514 23519 raft_consensus.cc:740] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.696951 18448 rpcz_store.cc:276] 0709 12:18:59.603183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.151597 (+548414us) service_pool.cc:224] Handling call
0709 12:19:00.686568 (+534971us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003b1ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:00.686687 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338167168}
I20260709 12:19:00.688004 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2526) took 2185 ms. Trace:
I20260709 12:19:00.697994 18440 rpcz_store.cc:276] 0709 12:18:58.502353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.505915 (+  3562us) service_pool.cc:224] Handling call
0709 12:18:58.506219 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.028687 (+522468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.687545 (+1658858us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54002aa654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.687853 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126416128}
I20260709 12:19:00.681941 24011 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.699534 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.700196 18446 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.702355 19041 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.705694 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.707036 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:00.711634 18300 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:00.718490 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.720168 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.721201 18448 raft_consensus.cc:2410] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:19:00.723397 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:00.686560 24245 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.648761 18447 pending_rounds.cc:85] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 6
I20260709 12:19:00.729159 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.731809 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.734817 18291 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:00.723472 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.731624 24158 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:00.738034 18443 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:00.738445 24158 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.739085 24158 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:00.739352 24255 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:19:00.687873 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.733215 18447 pending_rounds.cc:107] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:19:00.753332 18447 raft_consensus.cc:2889] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.759874 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 993) took 1013 ms. Trace:
I20260709 12:19:00.760176 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.760273 19038 rpcz_store.cc:276] 0709 12:18:59.746905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.876377 (+129472us) service_pool.cc:224] Handling call
0709 12:18:59.884276 (+  7899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.360894 (+476618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.741280 (+380386us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54006be4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.759806 (+ 18526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079397120}
I20260709 12:19:00.761090 19043 raft_consensus.cc:2393] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:00.761564 18440 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:19:00.736191 24235 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.501s	user 0.003s	sys 0.000s
I20260709 12:19:00.764927 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:00.766065 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 214921us
I20260709 12:19:00.669636 24214 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:00.733655 24245 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.648980 18444 rpcz_store.cc:276] 0709 12:18:59.584808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.553963 (+969155us) service_pool.cc:224] Handling call
0709 12:19:00.554500 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.554918 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:00.565947 (+ 11029us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.642255 (+ 76308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.642304 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.642355 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.642431 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.642533 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6737,"prepare.queue_time_us":7569,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":9593,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":23112,"thread_start_us":67512,"threads_started":1}
I20260709 12:19:00.762616 24246 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.776217 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:00.778322 19037 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:19:00.733500 24159 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:00.749518 23959 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.782501 18448 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:00.778661 18292 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:00.787315 23974 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:00.787643 24236 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.555s	user 0.004s	sys 0.000s
I20260709 12:19:00.749089 24233 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:00.802098 24202 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.000s
I20260709 12:19:00.790917 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.802685 24202 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.803743 19036 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
W20260709 12:19:00.811182 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.811465 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141970us
I20260709 12:19:00.812510 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:00.814293 19039 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:00.812530 18263 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.734316 23962 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:00.818480 24197 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.003s
I20260709 12:19:00.799988 24246 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.800211 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.819437 18295 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:19:00.812655 19042 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:00.820134 24074 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.820413 23974 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.820839 23962 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:00.820796 24074 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:00.818626 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:00.819948 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.775344 24245 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.769255 24214 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.799664 24025 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:00.824422 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.763051 23873 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:00.828784 18299 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:00.774624 24080 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:19:00.805311 24202 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:00.836302 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.834357 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2668) took 1036 ms. Trace:
W20260709 12:19:00.839223 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60662.5us
I20260709 12:19:00.823705 23959 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.829142 24025 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.830726 24214 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:00.853324 24232 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.619s	user 0.002s	sys 0.000s
I20260709 12:19:00.796327 23519 consensus_queue.cc:260] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.856096 23519 raft_consensus.cc:3060] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:00.859750 23938 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:00.861197 23974 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.862066 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.863196 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.864043 19038 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:19:00.864805 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.865196 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.863878 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.837726 18296 rpcz_store.cc:276] 0709 12:18:59.793422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.793674 (+   252us) service_pool.cc:224] Handling call
0709 12:18:59.817982 (+ 24308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.526579 (+708597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.526638 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.535719 (+  9081us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.576683 (+ 40964us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.609930 (+ 33247us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.697944 (+ 88014us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b18002f2360. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c68a32d70 00007f3c68a26293 00007f3c68a1bcea 00007f3c68a1c23a 00007f3c68e86a2a 00007f3c68e960bf 00007f3c68e96038
0709 12:19:00.698178 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.791437 (+ 93259us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54001cbbd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.791510 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:00.797424 (+  5914us) raft_consensus.cc:1685] finished
0709 12:19:00.797443 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.830141 (+ 32698us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51274,"prepare.queue_time_us":33205,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":6485,"raft.queue_time_us":26092,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":866,"spinlock_wait_cycles":1577830656,"thread_start_us":15566,"threads_started":1,"wal-append.queue_time_us":63756}
I20260709 12:19:00.869680 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.871762 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.872386 18444 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
W20260709 12:19:00.875432 24220 consensus_queue.cc:1175] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3da7ad8fa20a4591963b660e4c848e7f" responder_term: 17 status { last_received { term: 15 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: [16.12-16.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:00.876901 19043 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:00.878363 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.878523 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:00.879317 18443 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:00.879305 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.880095 18441 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:00.882217 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.884430 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.885140 19038 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:00.886276 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.886837 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1191) took 1178 ms. Trace:
I20260709 12:19:00.887631 19040 rpcz_store.cc:276] 0709 12:18:59.708389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.709004 (+   615us) service_pool.cc:224] Handling call
0709 12:19:00.886634 (+1177630us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b5400454a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:00.886767 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239954688}
W20260709 12:19:00.888842 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.873318 18446 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:00.892481 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:00.895298 18443 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:00.895586 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:00.896222 23873 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:00.899814 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2539) took 1823 ms. Trace:
W20260709 12:19:00.899923 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 56224.6us
I20260709 12:19:00.900431 18446 rpcz_store.cc:276] 0709 12:18:59.069568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.699063 (+1629495us) service_pool.cc:224] Handling call
0709 12:19:00.892376 (+193313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:00.902935 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.861999 23872 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.847631 24025 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:00.909687 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.913019 24222 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.913797 24222 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.914114 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.907191 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.906219 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.910048 23963 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:00.910238 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:00.913120 24074 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.909668 24201 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:00.915850 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.916910 18447 raft_consensus.cc:2410] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:00.905611 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.916791 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.915212 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.925949 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:00.926162 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.907224 24159 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:00.926807 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.927011 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.927350 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.927843 24159 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:00.928058 18447 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:00.928231 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.926046 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.929019 18441 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:00.927402 18443 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:00.928386 18442 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:00.929124 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.930841 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.929435 24166 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:00.933856 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.934641 24166 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:00.934863 23963 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:00.935298 23963 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.935750 24166 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.936344 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:00.936235 23963 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.937453 18446 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:00.929731 24222 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.938519 24026 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:00.938937 24026 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.939488 24026 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:00.939532 24247 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:00.941777 24096 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.942101 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:00.942256 24096 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:00.942761 24096 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:00.943693 24180 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.008s
I20260709 12:19:00.944167 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:00.944384 24201 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.939333 24076 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:19:00.945518 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.925984 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:00.946480 24222 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:00.946898 18440 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:00.947135 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.947852 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.948122 18444 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:00.921233 23872 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.948807 24197 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.949213 24197 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.949582 24197 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.951120 24201 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.951380 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.945561 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.937531 23963 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.951886 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:00.952317 24238 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.952766 24238 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.953147 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.953290 24076 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:00.903263 23938 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:00.954352 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:00.954911 24076 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.955842 24253 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.956856 24253 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.957234 24253 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.958729 23959 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:00.938964 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.959636 24076 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.960238 24076 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.960358 24215 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:00.953111 24238 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.960952 24215 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:00.960632 24237 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.955358 23938 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.961933 24215 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.928326 24159 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:00.963577 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.964357 18295 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:00.963922 24159 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.965315 24159 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.956215 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:00.930670 23883 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:00.946321 24202 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:00.967643 23821 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:00.968298 23821 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.968685 23936 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.969483 24238 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.970487 23821 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:00.951836 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:00.980113 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.975167 18447 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 11.
I20260709 12:19:00.907209 24011 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:00.967638 23883 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:00.982209 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 77447us
I20260709 12:19:00.983539 18449 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:00.983672 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:00.984110 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.983152 23883 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:00.976804 19036 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:00.986805 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.985697 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.987589 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:00.987634 19040 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:19:00.988121 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.990314 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:00.990861 19043 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:00.991074 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.991753 18441 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.990432 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:00.993148 19038 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:00.995103 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:00.997152 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45422.9us
I20260709 12:19:00.998806 18370 raft_consensus.cc:2368] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:19:00.976951 23936 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.008661 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 990) took 1255 ms. Trace:
I20260709 12:19:01.011700 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.012346 18300 raft_consensus.cc:2410] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:00.989955 24074 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:01.043382 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45492.7us
I20260709 12:19:01.010177 19041 rpcz_store.cc:276] 0709 12:18:59.744577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.744724 (+   147us) service_pool.cc:224] Handling call
0709 12:18:59.744993 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.498212 (+753219us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b540066eca0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df3dc0 00007f3c68e7e1c0 00007f3c68e7f231 00007f3c68e7bd8f 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c
0709 12:19:00.977172 (+478960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.977218 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.977272 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:00.985736 (+  8464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.990779 (+  5043us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.999992 (+  9213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5906,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":1726803200}
W20260709 12:19:01.020509 18294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.022s	sys 0.021s
W20260709 12:19:01.045226 18294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.022s	sys 0.022s
I20260709 12:19:01.045137 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:00.999310 23821 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:01.046159 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.047789 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.057353 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:00.992590 24201 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:19:00.988813 24197 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:01.064321 24133 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:19:01.065881 24197 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.066216 24142 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:01.060812 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.061255 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.062774 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:01.067665 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54231.2us
I20260709 12:19:01.051052 23821 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.066606 24142 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:01.046182 23883 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:01.071672 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62585.7us
I20260709 12:19:01.072355 18299 raft_consensus.cc:2368] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:01.049396 23722 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:01.067442 24208 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.005s	sys 0.022s
I20260709 12:19:01.101006 24158 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.129393 18449 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:01.056619 24222 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:01.128743 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:01.096515 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.083614 24180 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:01.137406 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43545.1us
W20260709 12:19:01.145810 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.138664 18296 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:19:01.139762 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.134099 19043 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:01.083899 24142 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.045312 24025 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.135038 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2543) took 1527 ms. Trace:
I20260709 12:19:01.073983 23821 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:01.146765 18448 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:01.102294 23919 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:01.152894 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.155962 18300 raft_consensus.cc:1240] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:01.102777 24011 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:01.130304 24237 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.159729 18443 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:01.129527 24208 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.008s	sys 0.029s
I20260709 12:19:01.136653 23872 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:01.148967 18445 rpcz_store.cc:276] 0709 12:18:59.404259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.404426 (+   167us) service_pool.cc:224] Handling call
0709 12:18:59.404733 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.895910 (+1491177us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:00.895953 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.896009 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:00.896037 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.903255 (+  7218us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400374e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:00.903313 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:00.931311 (+ 27998us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":468,"raft.queue_time_us":1597,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"spinlock_wait_cycles":3427162368}
I20260709 12:19:01.191362 19039 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:01.194331 18298 raft_consensus.cc:1275] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:19:01.207655 18298 pending_rounds.cc:85] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 11
I20260709 12:19:01.208027 18298 pending_rounds.cc:107] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:19:01.208341 18298 raft_consensus.cc:2889] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:01.194888 18441 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
W20260709 12:19:01.220681 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 135546us
I20260709 12:19:01.147185 24180 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:01.160110 23919 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.232514 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.183018 24214 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.236415 24163 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:01.236835 24163 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.237174 24163 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:01.238261 24214 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:01.239404 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.242444 18263 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.244184 18441 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:19:01.247634 24261 log.cc:927] Time spent T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:19:01.250471 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.259120 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:01.260273 24196 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.727s	user 0.014s	sys 0.000s
I20260709 12:19:01.255209 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2276) took 1167 ms. Trace:
I20260709 12:19:01.263293 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.261992 24245 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.269647 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.240540 23919 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:01.218662 23821 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.233013 24180 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.272941 18291 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:01.264391 18294 rpcz_store.cc:276] 0709 12:19:00.087447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.448649 (+361202us) service_pool.cc:224] Handling call
0709 12:19:00.463132 (+ 14483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.464043 (+   911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.464077 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.464725 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.045745 (+581020us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:01.045797 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.045888 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.255021 (+209133us) raft_consensus.cc:1685] finished
0709 12:19:01.255048 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.255158 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":580564,"threads_started":1,"wal-append.queue_time_us":696635}
I20260709 12:19:01.285244 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.287629 19035 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:19:01.289359 24013 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 12:19:01.174926 24253 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.254545 24133 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:01.289690 24204 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.269431 24247 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:01.300155 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:01.300652 19043 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:01.292876 23936 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:01.312391 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.314024 18295 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:01.304348 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.316006 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.294871 24205 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.319787 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.318390 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:01.320374 18441 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:01.326417 19039 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:01.327443 18298 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:01.303485 24011 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.313171 24247 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:01.311710 23936 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:01.369590 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:01.372629 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:01.339790 24204 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.379323 19040 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:01.390340 19039 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:01.369206 23936 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:01.323665 24026 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.325945 24133 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.400470 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.383679 23962 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.383908 24214 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:01.332829 24205 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.356762 24011 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.401465 23883 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.416152 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.399659 24197 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:01.438035 18371 raft_consensus.cc:2368] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
I20260709 12:19:01.440416 23883 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.469149 18441 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:01.469380 18293 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:01.468257 19043 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:01.475898 24215 consensus_queue.cc:1059] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:19:01.477182 18449 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:01.432623 24158 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.476470 24204 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.451140 24196 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.484597 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.415469 24076 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.418962 23519 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:19:01.438472 23962 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.479483 24245 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.487479 23884 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.452860 24197 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:01.497777 23821 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:01.506680 23938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.015s	sys 0.035s
W20260709 12:19:01.520985 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80366.2us
W20260709 12:19:01.524261 23938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.015s	sys 0.036s
I20260709 12:19:01.524868 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:01.509384 24158 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:01.494017 24214 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:01.524535 24267 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:19:01.535552 23136 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.514479 24253 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:01.529644 24264 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:19:01.497995 24196 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.488243 24247 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.529824 23883 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.551775 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:01.515969 23884 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.556954 23136 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
W20260709 12:19:01.572027 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 272574us
I20260709 12:19:01.556865 23938 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.543320 23963 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:19:01.590643 23136 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.591405 23136 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:01.556672 23821 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:19:01.570614 24026 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.574084 24133 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:01.560101 24163 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:01.617725 18246 auto_rebalancer.cc:893] Removing replace marker failed: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } } pending_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:01.619990 18246 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:19:01.625051 24272 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.003s
W20260709 12:19:01.625466 24273 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:19:01.617889 24282 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
W20260709 12:19:01.620395 18246 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:19:01.616101 24011 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:01.659107 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 254736us
I20260709 12:19:01.659726 24076 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:01.672581 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 116565us
I20260709 12:19:01.663741 24197 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:01.677736 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.679548 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.681919 18447 pending_rounds.cc:85] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 10
I20260709 12:19:01.690001 18441 raft_consensus.cc:2368] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16.
I20260709 12:19:01.690201 18447 pending_rounds.cc:107] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:19:01.690479 18447 raft_consensus.cc:2889] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:01.664736 24282 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.688002 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 390378us
I20260709 12:19:01.678215 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.668771 23962 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.696614 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.699446 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.707424 23962 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:01.707834 24253 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:01.708359 24253 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:01.709028 24253 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.708297 23962 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.712499 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:01.716184 23919 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:01.717639 24096 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.000s
I20260709 12:19:01.718035 24096 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.720021 24180 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.722332 24180 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.673403 24283 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:01.723300 24283 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:01.725521 24283 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.726272 18370 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:01.676584 24214 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.727399 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:01.730151 24276 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:19:01.731648 24278 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 12:19:01.733213 24281 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:19:01.696391 24159 consensus_meta.cc:287] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.731s	user 0.013s	sys 0.000s
I20260709 12:19:01.736434 23962 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.736799 24163 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.742287 23884 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:01.706243 24282 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.691039 24205 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.744665 24282 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.745029 24238 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:01.746501 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 47176.8us
I20260709 12:19:01.747316 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:01.748047 24133 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:01.748683 24133 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.751539 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:01.751789 24142 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.593s	user 0.006s	sys 0.005s
I20260709 12:19:01.752279 24133 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.752207 24142 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.752391 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:01.753261 18299 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:01.753924 18266 leader_election.cc:400] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:19:01.754235 18266 leader_election.cc:403] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.754297 24142 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.754772 24296 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.756897 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.756974 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.757514 23936 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.758152 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.757540 18296 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:01.758620 18291 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:01.759810 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.760377 18297 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.761888 18372 raft_consensus.cc:2368] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:19:01.763554 23936 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.764017 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:01.764565 19035 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:01.764638 18293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 2273) took 1678 ms (client timeout 1776 ms). Trace:
W20260709 12:19:01.765177 18293 rpcz_store.cc:269] 0709 12:19:00.086615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.091338 (+  4723us) service_pool.cc:224] Handling call
0709 12:19:01.764434 (+1673096us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b5400001754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:01.764560 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228689280}
I20260709 12:19:01.768291 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:01.769820 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:01.770745 24196 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.773566 23919 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.697975 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.775202 18293 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:01.775789 24096 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:01.776767 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.781425 24025 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:01.783823 24080 raft_consensus.cc:3055] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:01.784228 24080 raft_consensus.cc:740] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:01.785102 24080 consensus_queue.cc:260] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.786543 24080 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:01.786732 19035 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:01.787822 23714 consensus_queue.cc:1048] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:01.790153 18368 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:01.790208 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.793889 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:01.754959 24142 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.800176 18297 raft_consensus.cc:2410] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:19:01.796144 18296 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:01.704537 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.803381 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:01.805074 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.805711 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 93283.8us
I20260709 12:19:01.799340 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:01.809831 24233 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.817288 24293 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:01.817337 24233 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.817992 24293 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:01.818889 24293 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.820912 24233 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:01.834252 18374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.028s	sys 0.052s
W20260709 12:19:01.858996 18374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.028s	sys 0.052s
I20260709 12:19:01.865001 18293 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:01.862053 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47290.9us
I20260709 12:19:01.843484 24202 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:01.860597 24283 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:01.892511 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 202809us
I20260709 12:19:01.811851 24296 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
W20260709 12:19:01.908541 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86695.4us
I20260709 12:19:01.909109 18373 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:01.887406 24215 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.911747 18264 leader_election.cc:400] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16.
I20260709 12:19:01.916216 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:01.916877 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:01.919957 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:01.912168 18264 leader_election.cc:403] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.920822 18373 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.923700 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.923997 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:01.925081 19035 raft_consensus.cc:2410] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:19:01.920116 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 116849us
I20260709 12:19:01.926028 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.928289 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.929513 18368 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:01.907999 24166 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.930182 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:19:01.933835 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 229440us
I20260709 12:19:01.936375 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:01.946884 23883 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.950791 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.955626 18444 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.898348 24217 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:19:01.952062 18449 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:01.955843 18443 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:01.951435 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.910897 23962 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:01.954996 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.967411 18374 raft_consensus.cc:2410] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:01.957803 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:01.959012 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:01.963866 18449 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:01.969058 19041 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:01.927989 24286 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:01.972740 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.972632 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:01.974520 18446 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:01.973166 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.933250 24283 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:01.928355 23714 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:19:01.987660 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:01.972865 24180 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.969460 18449 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.954718 23963 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:01.975721 18368 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:01.994819 18412 leader_election.cc:400] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:02.002239 18412 leader_election.cc:403] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:01.998136 24291 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:19:01.956811 24096 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:02.005972 19035 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:19:02.003389 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 79787.3us
I20260709 12:19:02.004947 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:02.008668 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 209280us
I20260709 12:19:02.009204 18264 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:02.009433 18266 leader_election.cc:400] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
I20260709 12:19:02.009744 18266 leader_election.cc:403] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:02.012252 18444 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:01.968014 24297 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16.
I20260709 12:19:02.011490 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:01.995937 24159 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:02.013434 19007 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:01.964003 24166 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:01.956005 24294 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:19:02.027410 18371 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:02.037349 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.006114 24096 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.045902 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.046514 18293 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:02.024307 24238 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:02.000190 23962 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.025811 24159 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:02.026834 23883 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:02.026427 24011 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:02.097187 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.033015 23963 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:02.036727 24296 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.044843 24096 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:02.026727 24025 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 12:19:02.144163 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 101990us
I20260709 12:19:02.045789 24222 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:02.089402 24291 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:02.054296 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:02.156483 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.099433 23962 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.102094 24306 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 12:19:02.157706 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:02.177685 19035 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:02.099673 24159 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.100455 24166 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.164510 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.165337 18368 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.173682 24296 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
I20260709 12:19:02.174705 24025 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:02.175539 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 121288us
I20260709 12:19:02.049610 24197 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:19:02.047890 24217 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 12:19:02.148069 23883 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:02.183986 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.188270 24025 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.208252 19038 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
W20260709 12:19:02.186290 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 170361us
I20260709 12:19:02.193969 23714 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.212469 23714 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.211340 24025 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.214640 23714 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:02.217641 24253 consensus_meta.cc:287] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.003s
I20260709 12:19:02.218022 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:02.218695 18448 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:02.218204 24253 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.192570 23519 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:19:02.193589 23963 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.196442 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.227685 18366 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:02.199211 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.230890 18372 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.175288 24294 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:02.200774 24306 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:02.201174 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:02.202428 24202 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:02.205508 24166 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.205888 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:02.234114 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.159742 24283 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.253393 24142 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
W20260709 12:19:02.254187 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 46538.6us
I20260709 12:19:02.257872 24159 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.258759 24238 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.259351 24238 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:02.261873 18373 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
W20260709 12:19:02.299078 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97985.3us
W20260709 12:19:02.305653 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 71604.4us
I20260709 12:19:02.261504 24238 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:02.280622 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 74802us
I20260709 12:19:02.264938 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:02.308228 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:02.332396 18444 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:02.333524 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:02.267897 24300 log.cc:927] Time spent T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:19:02.240374 24294 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:02.307647 24215 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:02.305428 24233 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:02.249925 24166 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.281003 24076 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:02.338271 24215 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.265618 24075 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
I20260709 12:19:02.356619 24202 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.358337 24080 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:02.358361 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:02.343984 24166 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.346653 24159 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.279907 23883 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.412058 24306 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:02.424651 24306 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:02.427788 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69769.6us
I20260709 12:19:02.429075 18338 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:02.415652 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 150785us
I20260709 12:19:02.430799 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:02.431407 24309 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:02.431758 18412 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:02.432691 24217 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:02.433188 24217 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:02.434049 24217 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:02.356638 24011 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:02.435418 24308 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.435547 23946 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.435976 24308 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.436439 24308 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.436275 23946 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.438481 23946 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.440095 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:02.440685 24286 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:02.440790 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:02.441118 24286 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:02.442102 24096 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.442957 24294 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:02.441998 24286 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.444600 24294 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.443094 24025 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.445281 18449 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.446530 24286 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.415036 24253 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.404681 24080 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:02.431900 24309 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:02.448391 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.447688 24025 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.472776 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:02.456543 23722 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:02.457053 24080 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:02.464177 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 230696us
I20260709 12:19:02.466969 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:02.468653 24096 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.468750 24222 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:02.435623 24011 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:02.472740 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 164576us
I20260709 12:19:02.473619 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.362069 24215 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:02.417092 24238 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.484956 18443 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:02.486917 18292 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:02.482159 18263 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:02.481316 18449 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:19:02.508371 18449 pending_rounds.cc:85] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 12
I20260709 12:19:02.508711 18449 pending_rounds.cc:107] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:19:02.508996 18449 raft_consensus.cc:2889] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.488431 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.509826 18299 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.490252 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:02.490439 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.490361 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:02.510762 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.490888 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.512030 18294 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:02.491138 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.491751 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:02.512945 19043 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 12.
I20260709 12:19:02.447923 24076 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:02.427156 24306 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:02.404429 24202 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:02.490271 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.520633 19037 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:02.493336 23962 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.495380 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.526566 18298 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:02.495380 19035 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:19:02.499333 18373 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:02.499541 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:02.499812 19040 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:02.501886 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.503082 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.453167 24180 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:02.510802 19042 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:02.514235 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.498492 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.517957 19044 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:02.518692 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.518710 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.493045 24296 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.494814 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.516198 24293 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.519940 24076 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.526459 24202 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.481032 24253 consensus_queue.cc:1059] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:19:02.490124 24159 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.527437 24080 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.537289 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.558794 18369 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:02.559844 18368 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:02.562836 19039 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:02.517516 23938 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.503818 23722 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.566213 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:02.517168 24295 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:02.528326 23821 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:02.530015 18297 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:02.547621 18298 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:02.578908 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.583719 19044 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:02.563665 24075 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:02.524644 24096 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.565201 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:02.558076 24180 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:02.586381 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 65837.2us
I20260709 12:19:02.638803 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.579181 18296 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:02.639070 19038 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:02.538858 24233 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:02.546371 24308 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:02.644297 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116908us
I20260709 12:19:02.640789 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.645603 18292 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.545650 23962 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:02.583526 23821 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:02.649749 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.652680 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.635898 24295 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:02.650919 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.640782 24323 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:02.656272 18373 raft_consensus.cc:2393] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.640064 24180 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.583449 23963 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.658465 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.659003 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.572445 24309 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.586336 24159 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.661154 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.556191 24025 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 12:19:02.663081 18367 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:02.665246 23714 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:02.669072 24076 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.669009 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.569236 24296 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.682793 18299 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:02.570391 24293 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.689881 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.663196 18375 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:02.640362 24011 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.697009 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:02.669749 24315 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:19:02.700727 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.725013 19039 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:02.725364 18372 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:02.725958 18371 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:02.706846 18291 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:02.638687 23938 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.731879 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.723104 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.648509 24096 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.722658 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.735745 18374 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.736438 18294 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:02.737284 18372 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:02.727859 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.738152 18373 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20.
I20260709 12:19:02.739969 18367 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:02.697785 23714 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:02.741078 18370 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:02.741338 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:02.745003 18291 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:02.742434 18297 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:02.745505 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.746884 18375 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.750906 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.723590 24324 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.752313 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:02.753537 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 280810us
I20260709 12:19:02.700142 23884 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:19:02.753892 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:02.753602 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 114877us
I20260709 12:19:02.754841 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:02.755576 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.756757 24324 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.720269 23963 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:02.760100 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.762140 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.749377 24325 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:02.766507 24325 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:02.766167 18373 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 12: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.755911 24306 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:02.743893 24245 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:19:02.759671 24316 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.777751 18449 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:02.779066 18296 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.768066 24233 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:02.770618 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:02.762652 24317 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:02.738905 23821 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.784369 24332 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.793622 24332 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.795598 23936 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.796075 23936 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.780019 24233 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.797272 24233 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.785825 24322 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:02.797853 24320 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:02.800624 24309 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:02.802289 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.787542 24317 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.803283 24317 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.803648 24294 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.804170 24294 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.807822 24323 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:02.808223 24323 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:02.809005 24323 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.792472 19037 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:02.812800 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:02.813529 18291 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:02.815771 18375 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:02.800421 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.818023 18373 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.764750 24324 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:02.821075 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57439.1us
I20260709 12:19:02.822117 24294 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.822340 18296 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:02.771844 24306 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:02.822790 18339 leader_election.cc:336] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.934s (SENT)
I20260709 12:19:02.828377 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:02.810256 24332 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.825234 24322 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:02.771962 24217 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:19:02.838601 24322 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:02.839777 24322 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.827080 24330 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:02.826654 23821 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:02.845974 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.846524 18367 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.845235 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:02.847729 18373 raft_consensus.cc:2368] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:19:02.852662 24202 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.853732 19036 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:02.854804 24202 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:02.837774 24013 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:02.771301 23714 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.771275 24325 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.856556 23714 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.856873 24325 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:02.767819 24295 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.870472 24328 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:02.871026 24328 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.871958 24233 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.873195 24328 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.873015 19041 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:02.877760 18367 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:02.885800 18292 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:02.887836 23936 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.827713 24306 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:02.891882 24296 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.892911 24159 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.825724 24214 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:19:02.894937 24308 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.899518 24166 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.900825 18368 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:02.901377 23963 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.810261 24323 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.902171 23722 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.823380 24316 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.904654 24316 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.910322 18374 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:02.911784 24293 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.843750 24330 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:02.914589 24330 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:02.924814 23938 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.945159 24215 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.950443 24306 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:02.952008 24313 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.546s	user 0.003s	sys 0.000s
W20260709 12:19:02.956708 18415 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.701s (SENT) [suppressed 12 similar messages]
I20260709 12:19:02.952880 24215 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.959582 24080 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.962199 24080 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.965571 24080 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:02.967376 24306 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:02.973976 24096 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:02.975864 23883 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:02.835124 24317 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:02.996901 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.062366 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.063174 18449 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:03.064401 18443 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:03.066150 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:03.076052 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:03.077391 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.077905 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 322380us
I20260709 12:19:03.073360 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:03.079490 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:03.080185 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.081290 18443 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:03.082826 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.084513 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:03.084630 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:03.083578 19043 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:03.083633 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:03.087894 18372 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:19:03.087970 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 317695us
I20260709 12:19:03.088116 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:03.007988 23883 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:03.099005 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:03.090826 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:03.099946 19037 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 LEARNER]: Advancing to term 13
I20260709 12:19:03.095821 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.101876 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 12.
I20260709 12:19:03.102104 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:03.103572 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.104130 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.104612 18370 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:03.103645 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:02.894976 24180 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:03.105298 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 286404us
I20260709 12:19:03.108631 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.109573 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:03.009151 24309 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:19:03.117122 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.119777 19038 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:19:03.072304 24325 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:19:03.117758 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.123647 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.089972 24013 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:03.131093 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 174402us
I20260709 12:19:03.085825 23884 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:03.130568 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.089839 24238 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:03.085992 24294 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:03.097121 24329 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:03.135061 19039 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:03.134672 18369 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
W20260709 12:19:03.132838 18415 leader_election.cc:336] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.742s (SENT)
I20260709 12:19:03.106218 24253 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:03.072679 24096 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:19:03.118484 24159 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:03.153891 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.155256 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:03.156016 19043 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.156040 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.144857 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.111924 24166 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.150105 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:03.152261 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:03.098003 24075 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:03.161145 18446 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:03.168958 19037 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEARNER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:03.138788 24316 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.185731 18299 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:03.196300 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.196751 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.143610 24295 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:03.207556 19038 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:03.197419 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.195850 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:03.208565 18297 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:03.134805 24233 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:03.203693 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 70853.2us
I20260709 12:19:03.111593 24215 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:19:03.138406 24329 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.133977 24294 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.142572 24283 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:03.210335 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.141618 24196 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:03.145763 24013 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.224247 18444 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:03.225171 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:03.227574 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 119004us
I20260709 12:19:03.227798 18371 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.229007 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.229085 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.229876 18446 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:03.230005 19038 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:03.230820 24166 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:03.231187 24166 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.232573 19038 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:03.228284 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.244874 19042 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:03.211984 24202 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:03.333463 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85656.1us
W20260709 12:19:03.366168 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.366968 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:03.309676 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.346688 19043 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
W20260709 12:19:03.346765 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 118545us
W20260709 12:19:03.347826 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 112971us
I20260709 12:19:03.187974 24025 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:19:03.214509 24238 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.246011 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.207785 24295 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:03.362128 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 221401us
I20260709 12:19:03.205482 24308 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:03.223752 24294 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:03.377547 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.380081 19043 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:03.380568 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:03.381659 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.382965 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.346345 24283 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.223595 24329 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:03.252132 24286 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 12:19:03.353866 24330 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.407677 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:03.390892 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.397186 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.372720 24202 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:03.412607 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 166679us
I20260709 12:19:03.345585 24196 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.410410 18448 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:03.413729 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:03.221902 24316 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:03.386422 24324 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 12:19:03.212591 23883 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.398765 23946 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.436420 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.437394 18446 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:03.404759 24344 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.396831 24326 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:03.387054 24247 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:19:03.354267 24166 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.418601 24202 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.447856 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.448450 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:03.452486 18370 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.455596 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.434044 24334 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.432539 24309 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.399132 24308 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.472961 18368 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:03.475191 24308 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.435461 24317 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.476554 24317 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.435177 24343 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:03.433573 24335 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:03.451999 24297 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:03.416318 24330 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.451634 24076 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.479202 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71637.5us
W20260709 12:19:03.498523 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 381944us
I20260709 12:19:03.497985 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.499208 18446 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.433763 24324 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.456859 24344 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.480930 23821 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.486285 24222 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.500391 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:03.506338 18370 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:03.500272 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:03.506269 18443 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:03.514829 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.422936 24013 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.485150 24335 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.443568 23946 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.510293 23821 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:03.477136 24317 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.491139 24342 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:03.499787 24348 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.514889 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:03.569964 24222 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.571352 24346 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:19:03.574462 19043 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
W20260709 12:19:03.567422 24340 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:19:03.515879 23962 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:19:03.486738 24180 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.447175 24283 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:03.541714 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.575716 24339 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:19:03.602092 19041 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:03.600435 23821 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:03.576849 24338 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 12:19:03.508334 24076 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.614898 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.611159 18446 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:03.588922 24335 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.482030 24238 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:03.621524 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 106706us
I20260709 12:19:03.505188 24343 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.587983 24334 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.489320 24166 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:03.627717 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 127389us
I20260709 12:19:03.600642 18371 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
W20260709 12:19:03.630446 18442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.009s	sys 0.009s
I20260709 12:19:03.614976 24326 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.641012 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.623462 24217 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:03.643716 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 252895us
I20260709 12:19:03.652428 18446 raft_consensus.cc:2368] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
I20260709 12:19:03.652943 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:03.645134 18442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.011s	sys 0.009s
I20260709 12:19:03.644061 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:03.646255 18339 leader_election.cc:336] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.007s (SENT)
I20260709 12:19:03.649734 24317 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.654183 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.635663 24298 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:03.650434 24349 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:03.657675 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.669828 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.673133 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2583) took 1353 ms. Trace:
I20260709 12:19:03.674772 24316 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:03.657035 24323 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.675153 23946 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:03.676971 24357 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.680042 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.680814 18371 raft_consensus.cc:2410] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:03.677377 24357 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:03.622529 24348 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.716346 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:03.722800 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53049.4us
I20260709 12:19:03.724927 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:03.678087 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:03.671192 24217 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:03.644971 23884 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:19:03.645082 24308 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:03.668480 24317 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.800189 24317 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.802335 24348 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:03.647001 24334 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.803368 24355 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:03.774395 19044 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:19:03.803560 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.783510 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.799212 24324 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:03.800662 23714 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.944s	user 0.011s	sys 0.000s
W20260709 12:19:03.785827 24337 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 12:19:03.767225 24357 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:03.814815 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 136797us
I20260709 12:19:03.814386 24180 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.814970 24222 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.782305 24346 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:03.816418 24076 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.819361 24361 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:03.819800 23884 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:03.671692 24298 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.820711 24343 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:03.820791 24075 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:03.821486 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.822122 24308 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:03.826627 18446 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:03.873952 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:03.809119 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84267.3us
I20260709 12:19:03.874634 18446 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:03.847889 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:03.875468 19042 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:03.848011 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:03.777678 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 131464us
I20260709 12:19:03.673465 18442 rpcz_store.cc:276] 0709 12:19:02.319783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.331662 (+ 11879us) service_pool.cc:224] Handling call
0709 12:19:02.336687 (+  5025us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:02.342515 (+  5828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.342544 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:02.345715 (+  3171us) log.cc:844] Serialized 52 byte log entry
0709 12:19:02.481711 (+135996us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:02.509631 (+ 27920us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.979085 (+469454us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b180048ce80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:19:03.655636 (+676551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.655749 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.655805 (+    56us) raft_consensus.cc:1685] finished
0709 12:19:03.670294 (+ 14489us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.670442 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5662,"prepare.queue_time_us":23442,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":167629,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":8543,"spinlock_wait_cycles":14660864,"thread_start_us":801289,"threads_started":2,"wal-append.queue_time_us":180113,"wal-append.run_cpu_time_us":12294,"wal-append.run_wall_time_us":1036779}
I20260709 12:19:03.802335 24317 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:03.843900 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.645925 24344 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.812829 19041 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:03.873641 24180 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.878625 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.807407 24360 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:03.827168 24309 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:03.849375 24355 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.890812 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.870399 24330 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:03.872617 24075 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:03.894500 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:03.877485 24361 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.874327 24308 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:03.900317 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:03.901396 18446 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:03.879340 23884 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:03.882418 24346 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:03.884353 24076 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:03.885826 24360 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.877964 24202 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.920984 24355 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.903198 24328 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.928548 18442 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:03.913877 24360 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.933899 24180 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.938894 24298 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:03.930090 23821 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.827415 24326 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.947213 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:03.955368 23821 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.955828 24335 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.957875 24013 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:03.956489 24329 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.546s	user 0.004s	sys 0.006s
I20260709 12:19:03.958943 24329 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.958412 24013 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:03.960865 24013 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.963557 18443 raft_consensus.cc:2368] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:19:03.915655 24025 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:03.970795 24245 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.971396 24245 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.971891 24159 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:03.972378 24159 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:03.973707 24245 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.974993 24370 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:03.975600 24370 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.977823 24370 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.983968 24357 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:03.986187 24357 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:03.991343 23884 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:03.992051 18370 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:03.993191 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.993472 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.993988 18368 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:03.994580 18370 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:03.995659 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.997128 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.994895 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:03.929600 24328 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:03.998531 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 84698us
I20260709 12:19:04.002704 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:03.947599 24238 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:04.004808 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:03.983108 24025 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.005395 24355 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.007790 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 2326) took 1205 ms. Trace:
I20260709 12:19:04.007890 24357 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:04.008396 24328 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.009073 24025 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.009541 24334 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.011106 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:03.916075 24076 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.008253 18369 rpcz_store.cc:276] 0709 12:19:02.801553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.801981 (+   428us) service_pool.cc:224] Handling call
0709 12:19:04.006406 (+1204425us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:04.006540 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762506624}
I20260709 12:19:04.016705 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.017294 18370 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.021211 18366 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:04.022368 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.023098 18369 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:04.024148 18369 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:19:04.026762 18368 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:03.997771 24075 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.012639 24238 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.030423 24238 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.031733 24253 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:04.031870 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:04.032217 24253 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:04.033094 24253 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.034413 24253 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.036185 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.037892 18374 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:03.983311 24308 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.015954 24346 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.045148 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.045812 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:04.027318 24335 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.056165 24308 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.012149 24361 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.060154 24295 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:04.063472 24294 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.651s	user 0.005s	sys 0.005s
I20260709 12:19:04.063934 24294 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:04.064502 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.066239 18372 raft_consensus.cc:2368] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19.
I20260709 12:19:04.067818 24294 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.068071 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1391) took 1007 ms. Trace:
I20260709 12:19:04.068509 18447 rpcz_store.cc:276] 0709 12:19:03.060569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.071648 (+ 11079us) service_pool.cc:224] Handling call
0709 12:19:03.084288 (+ 12640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.160484 (+ 76196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.160530 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:03.160587 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:03.160632 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.067830 (+907198us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b54002030d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:04.067879 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.068008 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174584192}
I20260709 12:19:04.071858 18371 raft_consensus.cc:2410] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:19:04.072417 18415 leader_election.cc:336] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.969s (SENT)
I20260709 12:19:04.072903 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.076578 24142 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:04.078599 18374 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:04.078876 24142 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.050853 24360 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.082966 24322 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.083514 24142 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.085660 23883 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.051905 24159 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.098098 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.080941 24295 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.099658 18374 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.101257 24295 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.100266 18369 raft_consensus.cc:2368] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:19:04.103354 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.103559 24295 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.104036 18293 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:04.105050 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.105587 18293 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:04.106383 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.108136 18374 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:04.110019 24202 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.113309 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.113953 18293 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:19:04.115882 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1356) took 1025 ms. Trace:
I20260709 12:19:04.116488 18370 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:04.116254 18372 rpcz_store.cc:276] 0709 12:19:03.090503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.090633 (+   130us) service_pool.cc:224] Handling call
0709 12:19:04.114840 (+1024207us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b540030ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:04.115820 (+   980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2221254528}
I20260709 12:19:04.118630 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.119455 18366 raft_consensus.cc:2410] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:04.120173 24076 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.120373 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.121368 18292 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.118630 24329 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.122356 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.123051 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.123668 18366 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:19:04.125303 18299 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.128710 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.133929 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.134987 18295 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.140791 24360 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.144316 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:04.146145 24159 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.147099 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.147823 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:04.149010 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.149755 18297 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:04.150271 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:04.151104 18446 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:04.149776 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.154110 18443 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:04.155048 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.155632 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.156443 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.156816 24142 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.157303 18442 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:04.157727 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:04.158187 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.159098 24142 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.159112 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.160665 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.160701 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.161454 18300 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.161820 24335 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.164933 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.167169 24328 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.167936 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:04.168828 18447 raft_consensus.cc:2410] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:19:04.171098 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:04.173849 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.173228 18300 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:04.175552 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.176322 18446 raft_consensus.cc:2368] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:19:04.174552 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.177506 18374 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.179497 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.171550 24346 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:04.181550 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.182499 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.182938 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.183738 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.174157 24335 raft_consensus.cc:2764] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:04.181373 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.184973 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.185530 18447 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.185832 18446 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:04.186977 18339 leader_election.cc:400] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
I20260709 12:19:04.187282 18339 leader_election.cc:403] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:04.192150 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.193166 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.194861 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.195180 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.195478 19037 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.195788 19036 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.197289 18374 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:04.198503 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.192548 24295 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.199765 24334 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.200423 24334 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:04.200919 24334 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.201109 24215 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.201388 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.201619 24215 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.202092 24215 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.202289 18366 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:04.202932 23883 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.199826 24133 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.203400 23883 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.203853 23883 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.203662 24133 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.206115 24133 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.199991 24295 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.208276 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.209219 18445 raft_consensus.cc:2410] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:04.211452 24159 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.212383 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.213145 24159 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:04.213632 24159 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:04.214468 24159 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.215004 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.215786 18366 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:04.216507 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:04.208298 24295 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.182431 24346 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.217981 24346 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.218009 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.201093 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.220305 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.221127 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.221172 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.221416 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.222617 24202 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.222968 24202 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.223326 24202 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.223474 24133 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:04.224140 24133 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:04.224597 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.225059 24133 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.227053 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.234635 18442 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:04.236359 19036 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:04.238279 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.238821 18447 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:04.239553 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.240125 18374 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:19:04.241412 18374 pending_rounds.cc:85] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 12
I20260709 12:19:04.241724 18374 pending_rounds.cc:107] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:19:04.242018 18374 raft_consensus.cc:2889] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:04.243773 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.245524 18300 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:04.249197 19037 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:04.259087 18292 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:04.265807 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.266935 18299 raft_consensus.cc:2393] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.269824 18293 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:04.274143 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.247704 24360 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.275094 24361 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:04.276787 18292 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.276948 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.277837 18370 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:04.237094 24344 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15.
I20260709 12:19:04.278664 18295 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:19:04.280869 24013 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:04.269595 24096 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:04.283124 24013 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:04.284111 24013 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.284813 24334 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.285400 24013 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:04.289991 24334 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.297976 24202 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.308581 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.309667 19036 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:19:04.310201 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.282193 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:04.316355 18339 leader_election.cc:400] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:19:04.316884 18339 leader_election.cc:403] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:04.317595 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.318365 18296 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.320459 23136 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.320950 23136 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.324276 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.326567 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.329567 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.329622 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.330291 18339 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:04.318981 24013 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.317693 23519 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.335510 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.337765 18296 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:04.342615 23136 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.329142 24076 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:19:04.310098 23883 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.347786 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.355962 18296 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.348832 24334 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.332230 24298 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:19:04.351747 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2592) took 1214 ms. Trace:
W20260709 12:19:04.360195 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.332412 24283 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:04.373536 18449 rpcz_store.cc:276] 0709 12:19:03.137698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.137871 (+   173us) service_pool.cc:224] Handling call
0709 12:19:03.138177 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.247823 (+109646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.247861 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.195189 (+947328us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:19:04.195624 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.280532 (+ 84908us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:04.280580 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.280670 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.351447 (+ 70777us) raft_consensus.cc:1685] finished
0709 12:19:04.351465 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.351685 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4714,"raft.queue_time_us":87645,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":102769,"spinlock_wait_cycles":261946752,"thread_start_us":188964,"threads_started":2,"wal-append.queue_time_us":146102}
I20260709 12:19:04.335780 24013 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:04.384809 18296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:04.386842 18296 raft_consensus.cc:606] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:04.330327 24215 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.385710 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.388379 19038 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:04.330684 24346 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.394227 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.394649 18292 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:04.335011 24142 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:19:04.394454 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.341503 23519 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.401443 24142 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.401836 24142 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.380719 24013 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.402168 24142 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.402367 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.376075 24334 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.409924 24334 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.411376 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.414016 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.416692 24215 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.417630 19040 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:04.418772 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.420212 24202 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.386231 24283 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:04.422427 24283 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.425580 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.385605 24295 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.426178 18299 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.428928 24346 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:04.429927 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.430779 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.431325 24096 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:04.431632 18370 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:04.433063 18367 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:04.426514 24295 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.434520 18299 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:04.438900 19006 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.440164 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.441267 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.443861 18293 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.440395 24329 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:04.448980 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.452708 24323 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:04.453226 24323 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.439882 24346 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.458675 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.473781 19037 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:04.464622 24215 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.464536 24253 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:04.441104 24013 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.468044 18296 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:04.478317 24342 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.478763 24342 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.468369 24318 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:04.469269 23883 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:04.481336 24318 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.485258 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.486888 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.470636 18293 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:04.469743 24142 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.470826 24196 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.496400 24196 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.497098 24196 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.484350 24215 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.498335 24295 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.499455 24215 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.499953 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.500586 19039 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:04.501796 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.505122 24013 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:04.505867 18369 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:04.506359 24295 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:04.506798 24295 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.507201 24295 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:04.509034 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.509842 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.510648 19039 raft_consensus.cc:2368] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13.
I20260709 12:19:04.512416 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.514000 24011 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.514599 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.499006 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.515950 18298 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:04.516422 19042 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:04.516706 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.519075 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.526371 19037 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:04.471043 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:04.454288 24322 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:04.471688 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.535935 18295 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.453647 24323 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:04.447191 24202 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:04.493990 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.523746 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.540712 19042 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:04.539413 24346 raft_consensus.cc:2804] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.539676 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.561535 19035 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:04.561656 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 67754.2us
I20260709 12:19:04.562587 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.568114 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.568243 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:04.568004 19044 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:04.570245 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.570595 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.571656 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.572537 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.573956 24215 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:04.574400 19043 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:04.574536 24215 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:04.575616 24215 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.580698 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.580410 18338 leader_election.cc:400] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13.
I20260709 12:19:04.582170 18338 leader_election.cc:403] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:04.584044 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.584497 19009 leader_election.cc:400] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19.
I20260709 12:19:04.584792 19009 leader_election.cc:403] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:19:04.585291 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.586062 18298 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:04.590354 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.598242 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:04.541059 24308 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:04.608211 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.514640 24011 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 12:19:04.618254 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.579981 23946 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:04.567966 24346 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:04.580099 24361 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:04.580001 24013 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.540877 24329 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:19:04.587610 24096 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13.
W20260709 12:19:04.640836 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 154049us
I20260709 12:19:04.625792 19042 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:04.642293 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.653772 18293 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.653831 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.645754 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:04.654835 18370 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:04.568343 24202 raft_consensus.cc:697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:04.656749 24202 consensus_queue.cc:237] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.607510 24076 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.631384 24316 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:04.638548 24308 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:04.668107 18298 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:04.643536 24346 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.670181 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:04.668426 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:04.671217 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 152200us
I20260709 12:19:04.608084 24159 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:19:04.611294 24344 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:19:04.675979 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.680487 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.674225 24253 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:04.666652 24133 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19.
I20260709 12:19:04.709458 24253 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:04.716428 24328 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.710419 18293 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:04.716233 24253 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.676093 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:04.719813 24323 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.652820 24013 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.676636 24298 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.733650 18293 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.730340 24394 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:04.774175 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 252374us
I20260709 12:19:04.683602 24295 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.710668 24329 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:04.764804 23821 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.779306 23883 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:04.828836 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 193148us
I20260709 12:19:04.816149 24253 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.821944 24298 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.854589 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.856072 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.860214 18295 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:04.861106 18300 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [14.11-15.12]
W20260709 12:19:04.838145 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 165618us
I20260709 12:19:04.863991 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.829952 24142 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:04.865124 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 184687us
I20260709 12:19:04.865430 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.853168 24283 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.789456 24308 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.871116 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.871649 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:04.874230 19036 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.864079 24159 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.872495 18295 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:04.875283 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:04.875823 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.866333 24011 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:19:04.882107 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 213780us
I20260709 12:19:04.886492 18375 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:04.869632 24334 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.854946 24298 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:04.880570 24142 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.910950 19036 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:04.872869 24401 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:19:04.891602 24334 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.900144 23884 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:04.889508 24404 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.893841 24011 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:04.929610 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 53877us
I20260709 12:19:04.898056 23946 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:04.933051 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:04.918292 24142 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.919909 24159 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.908110 24076 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:04.926870 24398 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:19:04.888386 24415 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:19:04.930325 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:04.881548 24283 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:04.929646 24334 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.929090 24323 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:04.931545 24215 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:04.980419 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116563us
W20260709 12:19:04.982108 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53126us
I20260709 12:19:04.982257 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:04.933637 24013 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:04.937654 24142 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.990243 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:04.980062 24395 log.cc:927] Time spent T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:19:04.940203 24415 consensus_queue.cc:1243] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:04.980376 24404 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.999087 24011 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.003593 23946 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:04.975739 24420 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:04.989542 24159 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:04.939185 24317 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:19:04.998121 24295 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.036945 24425 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:05.039171 24425 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.040186 24295 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.040750 24295 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.038053 24404 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:05.050093 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:05.051990 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66244.4us
I20260709 12:19:04.990970 24215 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.053383 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:05.063709 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 130747us
I20260709 12:19:05.025055 24159 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:05.054838 24402 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:19:05.073899 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2702) took 1914 ms. Trace:
I20260709 12:19:05.078835 18293 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:19:05.070750 24403 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:19:05.081532 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:05.057447 24404 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.082842 18292 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:05.015516 24420 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:05.077984 24400 log.cc:927] Time spent T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:19:05.070470 24389 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:05.091070 24415 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:05.091765 24415 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.092877 24415 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.080191 24329 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:05.102892 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.077268 18294 rpcz_store.cc:276] 0709 12:19:03.150146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.150762 (+   616us) service_pool.cc:224] Handling call
0709 12:19:03.151292 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.151584 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:03.411810 (+260226us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.512009 (+100199us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.975950 (+463941us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b1800464260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:19:04.726097 (+750147us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b1800464260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:19:05.064393 (+338296us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.064466 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:05.064538 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.064629 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.064784 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23035,"prepare.queue_time_us":339721,"prepare.run_wall_time_us":41,"raft.queue_time_us":24445,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":187,"spinlock_wait_cycles":283508096,"thread_start_us":631334,"threads_started":3,"wal-append.queue_time_us":223829,"wal-append.run_cpu_time_us":1481,"wal-append.run_wall_time_us":1011483}
I20260709 12:19:05.117698 19036 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:05.089516 24298 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.125347 18292 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:05.095976 23884 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.132364 23884 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.099427 24428 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:05.145089 24431 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:05.146734 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93442.1us
I20260709 12:19:05.120412 24426 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:05.156800 24420 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.128942 24076 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:19:05.134783 24308 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:05.160262 24308 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:05.161957 24413 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:19:05.162324 24308 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.151263 24329 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:05.167398 24329 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.169567 24329 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.129292 24295 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.095152 24389 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.181573 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.182376 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.184000 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.126683 24407 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:05.184523 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.193849 18415 leader_election.cc:400] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:19:05.194187 18415 leader_election.cc:403] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:05.152782 24428 consensus_queue.cc:1243] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:19:05.127133 24159 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.172215 24283 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:05.208556 24342 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.002s
I20260709 12:19:05.198939 24215 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.171346 24404 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.209030 24342 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.145846 24431 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.177426 24426 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.213515 24431 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:05.167420 24433 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.192597 24405 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:19:05.215590 24433 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.218344 24433 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:05.205719 24346 consensus_meta.cc:287] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.526s	user 0.004s	sys 0.007s
I20260709 12:19:05.219545 24076 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.219138 24346 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.220008 24076 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:05.220897 24076 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.221896 24346 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.222090 24076 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.223158 24426 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:05.224859 24439 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 12:19:05.225835 24142 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.228500 24142 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.228547 24011 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:05.231824 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 49990.5us
I20260709 12:19:05.236038 24308 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:05.237360 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43643.7us
I20260709 12:19:05.241343 24428 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15.
I20260709 12:19:05.243546 24405 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.243928 24405 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.244843 24405 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.224776 24328 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:05.251892 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:05.269059 18294 raft_consensus.cc:2368] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:19:05.277070 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.279146 24405 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.279719 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.281731 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.322466 18294 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:05.308629 18292 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:05.324662 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.341881 18296 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:05.342696 18292 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:05.344266 18294 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:05.347306 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.267416 24438 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:19:05.350435 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:05.362349 18294 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:05.373296 24435 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:19:05.374200 18294 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
W20260709 12:19:05.408785 24417 log.cc:927] Time spent T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
W20260709 12:19:05.458633 24421 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.368s	user 0.003s	sys 0.000s
I20260709 12:19:05.509200 24425 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.519711 24075 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:05.524819 24423 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.453s	user 0.004s	sys 0.000s
I20260709 12:19:05.526018 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:05.531390 18297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:05.532047 18297 raft_consensus.cc:606] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:05.533370 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:05.534615 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.537547 18297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1f155085cb647ff957b468384227872"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:05.538069 18297 raft_consensus.cc:606] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:05.538344 24389 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.538388 18297 raft_consensus.cc:614] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:05.553850 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 315769us
I20260709 12:19:05.554257 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1116) took 1488 ms. Trace:
I20260709 12:19:05.554675 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:05.554673 18291 rpcz_store.cc:276] 0709 12:19:04.065610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.539964 (+474354us) service_pool.cc:224] Handling call
0709 12:19:04.540303 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.540704 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.540744 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.550488 (+1009744us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b2c000cd878. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:19:05.551043 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.552257 (+  1214us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:05.552297 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.552373 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.553964 (+  1591us) raft_consensus.cc:1685] finished
0709 12:19:05.553991 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.554199 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2363,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":18459,"spinlock_wait_cycles":7670528,"thread_start_us":779,"threads_started":1,"wal-append.queue_time_us":1214}
I20260709 12:19:05.583956 24342 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.598120 24429 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:05.598660 24429 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.599582 24429 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.600795 24429 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:05.601512 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2708) took 1092 ms. Trace:
I20260709 12:19:05.601797 18300 rpcz_store.cc:276] 0709 12:19:04.508811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.559946 (+ 51135us) service_pool.cc:224] Handling call
0709 12:19:04.860830 (+300884us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:04.862514 (+  1684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.601331 (+738817us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400090450. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:05.601464 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628003712}
W20260709 12:19:05.623795 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.625538 24361 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:05.625939 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71356.1us
I20260709 12:19:05.626328 24438 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.626084 24361 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.626693 24407 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:19:05.626179 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.626426 24404 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.629562 24404 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.630214 24407 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:05.630837 24449 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:05.626502 24436 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:19:05.631278 24449 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:05.631688 24404 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.631726 24436 consensus_queue.cc:1243] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:05.632143 24449 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.633206 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.634073 18300 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:05.634181 18298 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:05.626349 24075 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.627457 24342 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:19:05.635558 24075 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.636951 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.637737 19039 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:05.637943 24075 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:05.639091 24361 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.639688 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1363) took 1835 ms. Trace:
I20260709 12:19:05.640074 18371 rpcz_store.cc:276] 0709 12:19:03.804645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.805511 (+   866us) service_pool.cc:224] Handling call
0709 12:19:05.639445 (+1833934us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400116554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:05.639617 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446046464}
I20260709 12:19:05.640817 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:05.641413 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.642235 24438 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:05.642452 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.643400 24075 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:05.644037 18298 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 13: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:05.644996 24420 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.645942 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2602) took 1248 ms. Trace:
I20260709 12:19:05.646315 18369 rpcz_store.cc:276] 0709 12:19:04.397580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.397875 (+   295us) service_pool.cc:224] Handling call
0709 12:19:04.398142 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.506306 (+108164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.645704 (+1139398us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54000c8ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:05.645887 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247282048}
I20260709 12:19:05.645551 24420 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.643647 24438 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.647980 24438 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.648691 24420 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.649237 24420 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:05.649695 24420 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.650267 24438 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.643882 24075 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.651892 24420 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.652513 24075 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:05.653342 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.633440 24449 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:05.653834 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:05.650202 18440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2823) took 2268 ms (client timeout 2350 ms). Trace:
W20260709 12:19:05.655273 18440 rpcz_store.cc:269] 0709 12:19:03.381807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.883923 (+502116us) service_pool.cc:224] Handling call
0709 12:19:05.649974 (+1766051us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400376254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:05.650121 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594225024}
I20260709 12:19:05.656304 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.657373 19039 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:05.667451 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:05.658792 18263 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:05.670907 24394 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:05.671391 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:05.674194 19039 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:05.676666 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:05.678901 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.679677 19041 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:05.680521 24318 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.680342 24323 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:05.682099 18371 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:05.684793 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:05.691776 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.693188 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:05.673121 24394 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:05.683950 24075 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:05.701320 18369 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:05.702708 18371 pending_rounds.cc:85] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 7
I20260709 12:19:05.706045 18371 pending_rounds.cc:107] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:05.706316 18371 raft_consensus.cc:2889] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:05.706794 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.707278 18370 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:05.708139 24394 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.703945 24075 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.710943 24075 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:05.724169 18369 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:05.724112 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.728689 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:05.729707 18298 raft_consensus.cc:2368] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16.
I20260709 12:19:05.731411 24429 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:05.733711 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.735724 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.738482 24449 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.739806 24449 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.749485 24389 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.750303 24389 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.753279 24389 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.737226 24318 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:19:05.738626 24438 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:05.763717 18370 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:19:05.763423 24415 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.016s	sys 0.003s
I20260709 12:19:05.770735 24075 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:05.767966 24415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.016s	sys 0.003s
W20260709 12:19:05.773137 24011 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.003s
I20260709 12:19:05.782675 18298 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:05.783159 18444 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 12:19:05.784430 24437 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.639s	user 0.002s	sys 0.001s
I20260709 12:19:05.787470 24075 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.788013 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:05.791636 18366 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:05.798199 18372 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:05.800110 18366 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:19:05.808413 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:05.811172 24344 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:05.813894 24159 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:05.817153 24196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.006s	sys 0.001s
I20260709 12:19:05.815496 24361 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:19:05.823438 24159 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:05.826239 19040 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59426 (request call id 1351) took 1511 ms (client timeout 1740 ms). Trace:
W20260709 12:19:05.830799 19040 rpcz_store.cc:269] 0709 12:19:04.314977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.315146 (+   169us) service_pool.cc:224] Handling call
0709 12:19:05.825981 (+1510835us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b54004777d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:05.826159 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207722880}
I20260709 12:19:05.831921 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:05.832475 18298 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:05.832548 19040 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:05.834044 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:05.829792 24196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.007s	sys 0.001s
I20260709 12:19:05.840191 19040 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:05.841374 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:05.841424 24436 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:05.842810 24159 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:05.845876 24334 consensus_meta.cc:287] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.002s
I20260709 12:19:05.843551 24415 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:05.847183 24415 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:05.846403 24334 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.848057 24415 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.850354 24334 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:05.851624 19039 raft_consensus.cc:2368] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16.
I20260709 12:19:05.873523 24449 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:05.877475 24075 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:05.878932 24075 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:05.883771 18298 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:19:05.882625 19038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.026s	sys 0.014s
W20260709 12:19:05.886076 19038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.026s	sys 0.014s
W20260709 12:19:05.891376 24316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.000s	sys 0.037s
W20260709 12:19:05.921949 24316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.000s	sys 0.037s
I20260709 12:19:05.922209 24316 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:05.923149 24316 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.911082 24215 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:05.935652 24215 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.921398 24449 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:19:05.963091 24427 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.821s	user 0.004s	sys 0.010s
I20260709 12:19:05.969218 19044 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:05.973131 19044 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:05.968973 24011 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.975452 18448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2590) took 2844 ms (client timeout 2339 ms). Trace:
I20260709 12:19:05.975615 24011 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:05.975318 19044 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 6 to 8, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:05.975960 18448 rpcz_store.cc:269] 0709 12:19:03.131027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.131283 (+   256us) service_pool.cc:224] Handling call
0709 12:19:05.975251 (+2843968us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54002585d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:05.975352 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587597056}
I20260709 12:19:05.978124 24011 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:05.982250 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.967545 24449 consensus_queue.cc:1243] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer e23c0ce8d89940da8eac6a6002e349c5 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:05.988801 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:05.983596 24474 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:19:05.990597 18371 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:05.991163 18297 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:05.984442 24011 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:19:05.993885 18372 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:05.970625 24075 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:06.016489 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:05.983966 24361 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:05.985378 24159 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.983659 24405 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:19:05.983573 24335 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:06.025918 18374 raft_consensus.cc:2410] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:19:05.983423 24394 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:06.031107 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.038283 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1235) took 1202 ms. Trace:
I20260709 12:19:06.042182 19042 rpcz_store.cc:276] 0709 12:19:04.836055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.836199 (+   144us) service_pool.cc:224] Handling call
0709 12:19:04.836850 (+   651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.851934 (+ 15084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.852500 (+   566us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.876138 (+ 23638us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.044039 (+167901us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:05.044095 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.044190 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.962521 (+918331us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:06.037898 (+ 75377us) raft_consensus.cc:1685] finished
0709 12:19:06.037946 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.038210 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14261,"raft.queue_time_us":52193,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":11139,"spinlock_wait_cycles":39366656,"thread_start_us":122495,"threads_started":1,"wal-append.queue_time_us":198345}
I20260709 12:19:06.047750 18292 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [14.11-15.12]
I20260709 12:19:06.030254 24075 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.023681 24449 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:06.060104 24449 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:05.981419 24322 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Committing config change with OpId 11.8: config changed from index 6 to 8, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:06.056272 24215 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.058492 24436 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.067014 24436 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.029657 24159 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.066239 23136 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:06.063098 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.068580 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.068452 23136 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.078662 24429 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:06.079201 24429 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.080193 24436 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:06.081411 24429 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.083411 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.087386 24318 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.088071 24318 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.088730 23136 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:06.092168 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.096438 24075 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.096967 18448 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:06.097312 18443 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:06.099489 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.099869 24159 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.100137 18442 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.029989 24361 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.102089 24361 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.097409 18446 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:06.097332 18448 pending_rounds.cc:85] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 9
I20260709 12:19:06.104359 18448 pending_rounds.cc:107] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:06.104589 18448 raft_consensus.cc:2889] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:06.090909 24318 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:06.105476 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.105870 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.106448 18443 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:06.108232 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.108865 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.109090 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.109701 18443 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:19:06.109658 18449 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:06.110489 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.099596 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.030910 24335 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.123160 18442 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:06.207885 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.116106 18440 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:06.214462 18448 raft_consensus.cc:3060] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.166446 18449 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
W20260709 12:19:06.187355 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53731.9us
I20260709 12:19:06.215973 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.142850 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.218809 18446 raft_consensus.cc:2376] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17. Re-sending same reply.
W20260709 12:19:06.167760 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 118067us
I20260709 12:19:06.221189 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:06.221942 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.184060 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.217139 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.223253 18442 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.077697 24344 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.179589 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.226200 18449 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:06.227432 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.227972 18291 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
W20260709 12:19:06.184162 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.226490 18292 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:06.171985 24482 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:19:06.231029 18292 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:06.231894 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.238006 18443 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
W20260709 12:19:06.143107 24481 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:06.232067 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.111502 24429 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:06.192173 24415 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.140089 24075 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:06.241433 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.115623 24318 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:06.180303 24361 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:06.245345 24318 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.247397 18370 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:06.248095 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.143307 24323 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:06.249234 19040 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:06.250761 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.251744 18374 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:06.253005 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:06.232156 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.253613 18292 raft_consensus.cc:2376] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16. Re-sending same reply.
W20260709 12:19:06.252064 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 139345us
I20260709 12:19:06.245543 24361 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.256275 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.256728 18442 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:06.257584 18440 raft_consensus.cc:2410] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:19:06.246318 24415 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.259907 24318 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.255530 24394 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:19:06.260993 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.261575 18441 raft_consensus.cc:2376] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17. Re-sending same reply.
W20260709 12:19:06.262042 18441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2822) took 2880 ms (client timeout 1781 ms). Trace:
W20260709 12:19:06.262383 18441 rpcz_store.cc:269] 0709 12:19:03.381485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.398705 (+ 17220us) service_pool.cc:224] Handling call
0709 12:19:06.260840 (+2862135us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:19:06.261975 (+  1135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19972864}
W20260709 12:19:06.264312 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.260553 24415 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.265398 18412 leader_election.cc:400] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16.
I20260709 12:19:06.265628 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.267128 18447 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:06.267611 18412 leader_election.cc:403] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:06.238713 24344 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.290359 18372 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:06.244859 24429 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.294194 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.298622 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.247577 24075 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.252295 24323 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:06.319657 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800363e60 after lost signal to thread 17160
I20260709 12:19:06.295615 24318 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:19:06.325364 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800363880 after lost signal to thread 18232
I20260709 12:19:06.325825 24075 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.325978 18443 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:06.322006 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.329746 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.323974 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:06.333238 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.290424 24415 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.352087 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.319892 24492 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:06.357533 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67499.3us
I20260709 12:19:06.367100 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.367790 18441 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.379253 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.385548 18441 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:06.386905 24361 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:06.389758 18448 raft_consensus.cc:2468] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:06.391263 18446 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:06.396390 23136 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.397089 23136 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.376355 24497 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:06.406857 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.408475 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.408567 24500 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.409260 18446 raft_consensus.cc:2393] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:06.409225 24500 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.412276 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.412855 19041 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:06.414149 23136 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.428642 24429 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.430301 23519 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:06.431842 24335 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.433033 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.433178 24500 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.441428 18299 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:06.443792 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:06.445876 24316 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:06.446419 24316 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:06.447377 24316 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.451210 24498 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:06.451941 24344 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.452584 24496 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:06.451869 24498 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:06.461395 24076 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.239s	user 0.011s	sys 0.005s
I20260709 12:19:06.466279 24498 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.466943 24497 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:06.467351 24497 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:06.468111 24497 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.374598 24492 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.471084 19006 leader_election.cc:400] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16.
I20260709 12:19:06.471594 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.473675 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.472283 18446 raft_consensus.cc:1275] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 16. (term mismatch)
I20260709 12:19:06.474860 18446 pending_rounds.cc:85] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 15
I20260709 12:19:06.475143 18446 pending_rounds.cc:107] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 17.16
I20260709 12:19:06.475482 18446 raft_consensus.cc:2889] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:06.478776 23946 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:06.483913 24492 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.485086 24323 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.486367 24323 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:06.487103 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:06.487607 18367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2603) took 2086 ms (client timeout 1535 ms). Trace:
W20260709 12:19:06.488052 18367 rpcz_store.cc:269] 0709 12:19:04.401684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.402117 (+   433us) service_pool.cc:224] Handling call
0709 12:19:06.487435 (+2085318us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b540034a654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:06.487539 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68591872}
I20260709 12:19:06.489617 24394 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:06.490223 24394 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.492528 24394 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.493606 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.497450 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.502455 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.503057 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.503365 18447 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:06.509235 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.509298 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.511624 18446 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:06.512688 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.513305 18448 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:06.515165 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:06.516672 24202 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:06.517288 24202 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:06.518337 24202 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.501518 24415 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16.
I20260709 12:19:06.527714 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.528498 18291 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:06.531562 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.532321 18448 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
W20260709 12:19:06.550653 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 191789us
I20260709 12:19:06.551613 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.552551 18448 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:06.553794 24449 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.558749 24449 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:06.560883 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:06.561745 19036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.381s	user 0.008s	sys 0.017s
I20260709 12:19:06.555102 24335 raft_consensus.cc:993] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:06.565403 24196 raft_consensus.cc:1081] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:06.565459 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:06.568130 24505 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:06.568574 24505 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.568918 24505 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.577112 24505 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.579649 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:06.580660 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.581288 19036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.009s	sys 0.017s
I20260709 12:19:06.582391 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1026) took 1403 ms. Trace:
I20260709 12:19:06.582849 19036 rpcz_store.cc:276] 0709 12:19:05.179319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.179559 (+   240us) service_pool.cc:224] Handling call
0709 12:19:05.179991 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.180267 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:05.180611 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.581908 (+1401297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.581984 (+    76us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:06.582042 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.582130 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.582280 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"thread_start_us":1400856,"threads_started":1,"wal-append.queue_time_us":197468,"wal-append.run_cpu_time_us":1438,"wal-append.run_wall_time_us":1025217}
I20260709 12:19:06.585163 24503 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.586652 24505 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.564318 24449 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:06.595937 18291 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:06.603555 18367 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:19:06.604143 18367 pending_rounds.cc:85] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 12
I20260709 12:19:06.604411 18367 pending_rounds.cc:107] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:19:06.604692 18367 raft_consensus.cc:2889] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:06.607535 24449 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:06.614460 19035 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:06.614039 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.612098 18368 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:06.619630 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.620996 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:06.622427 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.613312 24407 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.623878 24407 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:06.624617 24316 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:06.624773 24407 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.625890 24407 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:06.627023 18321 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8760 after lost signal to thread 18324
W20260709 12:19:06.627440 18321 debug-util.cc:398] Leaking SignalData structure 0x7b08003c87e0 after lost signal to thread 18325
I20260709 12:19:06.621554 24360 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:19:06.628504 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:06.621398 24415 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:19:06.631012 18443 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:06.632612 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.612043 24449 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.632922 24133 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:06.634855 23883 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:06.621749 24404 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:19:06.638111 18297 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:06.663025 18299 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:06.639839 24394 consensus_queue.cc:1048] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:19:06.640285 18444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2819) took 3264 ms (client timeout 2144 ms). Trace:
W20260709 12:19:06.664112 18444 rpcz_store.cc:269] 0709 12:19:03.376324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.376490 (+   166us) service_pool.cc:224] Handling call
0709 12:19:05.525881 (+2149391us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:19:06.640069 (+1114188us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:06.640198 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610720512}
I20260709 12:19:06.641275 18445 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:19:06.639060 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:06.665561 18445 pending_rounds.cc:85] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 12
I20260709 12:19:06.665834 18445 pending_rounds.cc:107] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:19:06.666082 18445 raft_consensus.cc:2889] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:06.644528 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.645645 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.649395 18442 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:06.653575 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.655733 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.668553 18295 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:06.657012 18440 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:06.644263 24133 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.660280 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:06.669448 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:06.668992 18448 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:06.679999 23136 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:06.680447 23136 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:06.680852 23136 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.681959 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:06.687093 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:06.688565 18447 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.695804 23519 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:06.696305 23519 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:06.687484 24133 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:06.655349 24493 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.327s	user 0.004s	sys 0.000s
I20260709 12:19:06.698338 24322 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:06.700536 23519 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:06.703958 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:06.689160 24196 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:06.711410 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.712949 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.716125 24410 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.716626 24410 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.718762 24410 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.718890 18367 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:06.721243 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:06.721921 18294 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:06.723022 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.723611 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:06.725471 24410 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:06.726082 24410 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.730338 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.726635 24410 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:06.714169 24323 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:06.732967 18447 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:06.735780 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.742378 18366 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:06.742511 18375 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:06.733083 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:06.743888 18367 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.710403 24196 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.746443 18412 leader_election.cc:400] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:19:06.748948 18412 leader_election.cc:403] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:06.749727 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1234) took 1914 ms. Trace:
I20260709 12:19:06.750412 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.751657 18366 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:19:06.757612 18367 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:06.707491 24404 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:06.713882 24415 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.747884 18375 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:19:06.759018 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800327340 after lost signal to thread 18399
I20260709 12:19:06.754338 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.750193 19037 rpcz_store.cc:276] 0709 12:19:04.835679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.836285 (+   606us) service_pool.cc:224] Handling call
0709 12:19:04.836807 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.837191 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.837229 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.855720 (+ 18491us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.967526 (+111806us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:04.967587 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.967686 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.524231 (+556545us) spinlock_profiling.cc:243] Waited 91.8 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:06.745201 (+1220970us) raft_consensus.cc:1685] finished
0709 12:19:06.745233 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.749665 (+  4432us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37132,"raft.queue_time_us":36701,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":6325,"spinlock_wait_cycles":211103360,"thread_start_us":95799,"threads_started":1,"wal-append.queue_time_us":148127,"wal-append.run_cpu_time_us":6458,"wal-append.run_wall_time_us":1511234}
W20260709 12:19:06.747076 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 116559us
I20260709 12:19:06.760171 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:06.764496 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.764966 18374 raft_consensus.cc:2410] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:19:06.750253 24196 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.748631 24405 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:06.753770 24323 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:06.777952 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.783958 24415 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.771961 24011 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.778579 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 2356) took 1071 ms. Trace:
I20260709 12:19:06.747818 23714 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:19:06.760555 24318 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:19:06.792520 18373 rpcz_store.cc:276] 0709 12:19:05.706791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.762231 (+1055440us) service_pool.cc:224] Handling call
0709 12:19:06.778535 (+ 16304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.758387 24308 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:19:06.799757 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:06.810920 19037 raft_consensus.cc:606] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:06.814779 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 132926us
I20260709 12:19:06.844482 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.845180 18373 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:06.828058 24410 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.859371 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.850492 23136 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:06.860096 18372 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:06.860496 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.861459 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:06.874562 18368 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.862879 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.864941 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:06.876720 19037 raft_consensus.cc:606] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:06.855512 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:06.863536 18321 debug-util.cc:398] Leaking SignalData structure 0x7b080032ae20 after lost signal to thread 18473
I20260709 12:19:06.864212 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:06.886361 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.888284 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.889673 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:06.890765 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:06.891605 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 94091.6us
I20260709 12:19:06.891341 18372 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:06.895179 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:06.864892 19007 connection.cc:256] RPC call timeout handler was delayed by -1.5880484230001457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2581s for a response
I20260709 12:19:06.897539 23136 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:06.898682 18266 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:06.899119 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:06.899822 18300 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:06.855850 24442 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.909966 18368 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:06.857788 24342 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:06.865809 24496 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.920230 24496 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:06.924796 24496 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:06.871341 24410 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:06.932410 24394 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:06.859580 24323 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:06.939368 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.905488 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:06.917721 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:06.942782 23136 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:06.943128 23136 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:06.943483 23136 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.952636 23136 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:06.958958 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.982625 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.969151 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:06.901957 23883 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:06.911791 24510 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:19:06.910373 24442 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:06.915827 24342 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:06.910670 23946 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:06.861505 24316 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:19:07.053812 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:07.068931 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:07.074445 24518 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:07.076169 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:06.894164 24133 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.079210 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:06.866303 23519 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:07.087551 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800305d40 after lost signal to thread 19068
W20260709 12:19:07.138829 18321 debug-util.cc:398] Leaking SignalData structure 0x7b0800108c80 after lost signal to thread 19069
W20260709 12:19:07.139639 18321 debug-util.cc:398] Leaking SignalData structure 0x7b080028c860 after lost signal to thread 24265
W20260709 12:19:07.140108 18321 debug-util.cc:398] Leaking SignalData structure 0x7b080000f2e0 after lost signal to thread 24327
I20260709 12:19:07.087608 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:07.093109 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 328661us
I20260709 12:19:07.145980 19043 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
W20260709 12:19:07.129149 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 160079us
I20260709 12:19:07.128165 24283 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.097365 24322 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.148568 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:07.146486 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.128298 24308 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.128472 24159 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.097659 23883 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:07.134966 24515 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:19:07.151389 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.121335 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:07.178726 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 363249us
I20260709 12:19:07.183005 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:07.136886 24316 consensus_queue.cc:1243] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:19:07.147284 19043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 1029) took 1615 ms (client timeout 1566 ms). Trace:
I20260709 12:19:07.139509 23946 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:07.199995 18264 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:07.193279 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:07.194023 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72805.7us
I20260709 12:19:07.198734 18366 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:19:07.189527 19043 rpcz_store.cc:269] 0709 12:19:05.532275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.087262 (+1554987us) service_pool.cc:224] Handling call
0709 12:19:07.147198 (+ 59936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.175733 24415 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:07.228696 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 331012us
W20260709 12:19:07.229456 19007 connection.cc:256] RPC call timeout handler was delayed by -1.504956975000141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 12:19:07.230108 19007 connection.cc:256] RPC call timeout handler was delayed by -1.146939231000033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 12:19:07.230945 19007 connection.cc:256] RPC call timeout handler was delayed by -1.1039166170000954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
I20260709 12:19:07.217228 18374 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 6 to 8, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:07.233870 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.234712 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.190392 23883 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:07.204252 24405 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.153393 24283 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.246973 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.247761 18445 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:07.153426 24322 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.284977 24322 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.180523 24429 consensus_queue.cc:1059] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:19:07.247893 18300 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:07.236567 24407 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.239571 24075 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:07.286013 24429 raft_consensus.cc:3055] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.248988 24525 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:07.244992 24318 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:19:07.183988 24442 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.287185 24011 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:07.286056 24407 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.287779 24011 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:07.287814 24442 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.288549 24133 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.288739 24011 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.289520 24407 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.289949 23519 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.289954 24011 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:07.290612 23519 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.290992 23519 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:07.291446 23519 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:07.291297 24520 consensus_queue.cc:1175] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3da7ad8fa20a4591963b660e4c848e7f" responder_term: 17 status { last_received { term: 15 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:07.281315 24323 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:07.281503 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46889.6us
I20260709 12:19:07.293704 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.294833 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.232019 24342 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.295593 24318 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:07.181180 24308 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.286487 24429 raft_consensus.cc:740] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:07.295965 24496 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:07.286993 24525 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:07.296928 24429 consensus_queue.cc:260] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.297648 24308 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.298139 24429 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.298424 18449 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:19:07.297292 24525 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.298768 18449 pending_rounds.cc:85] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 11
I20260709 12:19:07.299072 18449 pending_rounds.cc:107] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:19:07.299332 18449 raft_consensus.cc:2889] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.191061 24196 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.300177 24308 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.301923 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.214933 24522 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.184250 24159 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:07.306838 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60587.4us
I20260709 12:19:07.304826 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:07.305274 24405 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.311874 24510 raft_consensus.cc:993] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:07.312371 24467 raft_consensus.cc:1081] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:07.314474 24283 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.301658 24518 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.317233 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.305634 24196 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.305014 24522 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:07.317607 24415 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:07.321403 24519 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.321530 24522 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.322332 24202 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:07.322731 24202 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:07.323571 24202 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.308292 24335 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.325513 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.334498 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:07.319001 24518 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:07.337208 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.337507 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.338960 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.339322 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.321662 24415 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:07.287303 23946 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.326495 24335 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:07.342851 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.339620 19041 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:07.353745 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:07.358132 19035 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:07.358842 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.341300 19036 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:07.359433 19044 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:07.333320 24467 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:07.270428 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:07.363374 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.362596 19041 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:07.421862 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.447089 19038 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:07.337267 24518 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.464114 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:07.354635 24415 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:07.470314 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 45228.9us
W20260709 12:19:07.478608 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.496821 19038 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:07.497529 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.361505 24335 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.362246 24308 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:07.522701 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.524731 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.354488 24322 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.155s
I20260709 12:19:07.418888 19035 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:07.359623 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:07.531574 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 172012us
W20260709 12:19:07.360983 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 212464us
W20260709 12:19:07.522442 24502 log.cc:927] Time spent T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:19:07.518316 24438 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.499504 23714 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:07.540562 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:07.409525 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.539551 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:07.573761 19044 raft_consensus.cc:1240] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.579838 24283 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.577881 24438 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.583464 24438 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.590514 19038 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:07.600783 24529 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:07.601466 24529 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.542119 24519 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.610713 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.611361 18447 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:07.614230 18443 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:07.606389 24308 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.616297 24308 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.616827 18441 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:07.618646 24308 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.619544 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.575922 24215 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.606921 24438 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.606616 24510 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:07.609666 24519 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.633239 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.628006 24215 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.626736 23883 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.625617 23519 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.623963 24159 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.637033 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:07.642539 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.661706 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.662933 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:07.664430 18448 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:07.665752 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:07.666723 18446 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:07.636461 19044 raft_consensus.cc:2410] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:19:07.640393 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.671481 18449 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:07.642128 24510 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:07.654809 18371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.009s	sys 0.016s
W20260709 12:19:07.673519 18371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.010s	sys 0.016s
I20260709 12:19:07.641371 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:07.677310 19042 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:07.677412 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:07.677831 18442 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:46956
I20260709 12:19:07.681509 18444 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:07.681980 18447 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:19:07.687608 24533 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:19:07.691884 24532 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:19:07.607053 23714 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.705118 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.652072 24529 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.635182 24519 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.702437 24536 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:19:07.673599 24510 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.679045 24133 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:19:07.693423 24323 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 12:19:07.718901 18264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.018s
W20260709 12:19:07.766006 18264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.013s	sys 0.020s
I20260709 12:19:07.675123 24467 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.778486 24215 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.779565 23883 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.780149 23883 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:07.780637 23883 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.782253 24534 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:07.782788 24534 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.783505 23883 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.784904 24534 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.785857 24215 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.790261 23714 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.780247 18440 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
W20260709 12:19:07.766793 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 566824us
I20260709 12:19:07.795022 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:07.796324 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.008s	sys 0.016s
W20260709 12:19:07.796729 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.008s	sys 0.016s
W20260709 12:19:07.797014 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 534968us
I20260709 12:19:07.802331 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.768275 24318 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:07.786217 23519 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:07.793237 24429 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.810684 23136 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.811371 18440 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.634186 24438 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.778563 24518 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.821811 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.803350 24529 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:07.830520 24529 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:07.831100 24529 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:07.833555 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.834736 18440 raft_consensus.cc:2410] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:19:07.824478 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:07.821995 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 144688us
I20260709 12:19:07.842451 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:07.826609 18443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:07.843886 18443 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:07.844314 18443 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.827781 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.835400 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:07.816640 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.845489 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.846493 18440 raft_consensus.cc:2393] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:07.850601 24467 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.857571 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:07.859040 18440 raft_consensus.cc:2376] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13. Re-sending same reply.
I20260709 12:19:07.860136 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.860246 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:07.861544 18440 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.818845 24318 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.863669 18442 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:07.874963 18442 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:07.854072 24522 consensus_meta.cc:287] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 12:19:07.822608 24429 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.876041 18443 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:07.849154 24410 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:19:07.850659 23714 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:19:07.894595 24318 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.892863 18440 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:07.892690 18442 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.892849 18443 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.862901 24442 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.904286 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:07.914163 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:07.935290 18300 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:07.939137 18444 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:07.942013 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:07.888926 24133 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:07.942863 18291 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:19:07.946815 24543 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:07.947888 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:07.948753 18291 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
W20260709 12:19:07.948925 24542 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:07.952934 18291 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:07.957443 23519 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:07.958119 23519 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:07.960682 23519 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.968180 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:07.974844 23519 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:07.975522 24525 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.976060 24525 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:07.978516 24525 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.979904 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:07.987316 24525 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.992095 24438 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:07.992194 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:07.995708 24429 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.996954 24518 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.997630 24525 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.938468 24442 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.000185 24442 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:07.895586 24522 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.001490 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.002298 24522 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:08.005240 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.005851 19038 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:08.006645 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.008482 24442 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.009496 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.010188 19041 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:08.010416 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.012140 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.012832 19042 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:08.018400 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.018442 24442 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.020054 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.021790 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.026813 24442 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:08.028182 18264 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.031586 24133 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.033231 24442 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:08.033713 24133 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:08.040789 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.043155 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.043795 18300 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:08.044904 19035 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:08.045845 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.046765 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.049255 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.049909 19038 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:08.054958 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.010895 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.059492 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.061925 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.066108 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.067184 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.067735 18443 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:07.986824 24415 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.056159 23884 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:08.033753 24442 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:08.074805 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.072098 24539 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
W20260709 12:19:08.088289 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 241109us
W20260709 12:19:08.073756 24335 consensus_meta.cc:287] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 12:19:08.064186 24522 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.107584 18449 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:08.116305 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.116921 18449 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:08.116781 24335 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.117870 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.118453 18449 raft_consensus.cc:2376] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17. Re-sending same reply.
I20260709 12:19:08.120102 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.120286 24335 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.120774 24522 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.121186 24522 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.120790 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.123219 19042 raft_consensus.cc:2393] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 13: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:08.124529 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.112568 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.113868 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.134747 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.135464 18449 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:08.135538 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.136133 18370 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:08.136442 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.136987 18449 raft_consensus.cc:2376] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17. Re-sending same reply.
I20260709 12:19:08.137852 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.138360 18449 raft_consensus.cc:2376] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16. Re-sending same reply.
I20260709 12:19:08.142369 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.143792 18443 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:08.145165 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.146101 18443 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 13: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:08.146705 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.080690 24529 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.105577 23884 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.146979 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.177088 18443 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.177249 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.178747 18370 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:08.175172 18444 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:08.176132 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.153059 24222 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:08.181207 24222 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:08.181427 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.182214 24415 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.182065 24222 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.182782 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.184165 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.185333 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.188546 18449 raft_consensus.cc:2376] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17. Re-sending same reply.
I20260709 12:19:08.189265 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.190271 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.191452 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:08.191867 18300 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:08.192112 18292 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:08.193569 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.194621 18298 raft_consensus.cc:2410] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:19:08.196266 18295 raft_consensus.cc:1275] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:08.199779 24510 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:08.201332 18298 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.203387 24531 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:08.203565 24442 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.205706 24535 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.206241 24535 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.206622 18298 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Committing config change with OpId 11.8: config changed from index 6 to 8, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:08.208403 24535 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:08.210041 24529 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.210233 24510 consensus_queue.cc:1048] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:08.099082 24519 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.212291 24222 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:08.212885 24519 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.213619 18441 raft_consensus.cc:2368] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:19:08.214314 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.215044 18444 raft_consensus.cc:2376] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17. Re-sending same reply.
I20260709 12:19:08.217360 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.220932 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.222084 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.221087 18444 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:08.219607 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.219527 18291 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:08.224380 18293 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:08.230450 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:08.231187 18300 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:08.231654 18300 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:08.219378 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.237722 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.238334 18293 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:08.239104 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:08.240350 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.241391 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.241942 19037 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.232743 18300 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.244561 18300 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.245326 18443 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:08.246646 18444 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.248376 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.248751 18444 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:08.249814 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.252141 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.252846 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:08.253077 18444 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:19:08.256047 18444 pending_rounds.cc:85] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 8
I20260709 12:19:08.214210 23884 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:08.259850 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.155217 24335 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.256800 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.257942 18444 pending_rounds.cc:107] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:19:08.177845 24322 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.270841 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:08.280602 18296 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.272123 18444 raft_consensus.cc:2889] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:08.277432 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.278317 19037 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:08.283044 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.286217 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:08.286420 18300 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:19:08.288164 18300 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:08.289750 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.290051 18447 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:19:08.246136 24519 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:08.285640 24415 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.295524 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:08.296640 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.299471 18447 pending_rounds.cc:85] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 12
I20260709 12:19:08.303520 18447 pending_rounds.cc:107] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:19:08.302532 18444 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:08.291169 24520 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.307924 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:08.308156 24556 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:19:08.308689 18444 raft_consensus.cc:2393] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:08.309664 18444 raft_consensus.cc:1240] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:08.269580 24531 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:08.311906 24531 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.312265 24531 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:08.313146 24531 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.314396 24531 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.291510 24322 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.293963 24438 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.315546 24342 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.315464 24438 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.317898 24438 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.296670 24429 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.318557 24429 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.318898 24429 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.319885 24202 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.321820 18296 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:08.306787 24405 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.323289 24405 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.325459 24405 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.303902 18447 raft_consensus.cc:2889] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.328969 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.302310 24415 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.287885 24335 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.330683 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:08.340461 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:08.345121 24415 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.340152 24563 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:08.342279 24520 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:08.361246 24438 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Committing config change with OpId 11.10: config changed from index 8 to 10, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:08.370026 18297 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:08.375681 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1387) took 1472 ms. Trace:
W20260709 12:19:08.379959 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.394274 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.395078 24449 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.396627 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.397586 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.398331 24283 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:08.399886 24497 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:08.401681 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.402292 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 255687us
I20260709 12:19:08.423852 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.424155 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.408555 18367 rpcz_store.cc:276] 0709 12:19:06.903523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.122737 (+219214us) service_pool.cc:224] Handling call
0709 12:19:07.123138 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.123387 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:07.608781 (+485394us) log.cc:844] Serialized 14 byte log entry
0709 12:19:08.375154 (+766373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:08.375256 (+   102us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:08.375330 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.375428 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:08.375597 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":575623,"prepare.queue_time_us":579092,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":548141,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":644334,"threads_started":2,"wal-append.queue_time_us":241751}
I20260709 12:19:08.428579 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:08.431861 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.442127 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45611.9us
I20260709 12:19:08.443464 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:08.450138 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.438494 18264 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:08.455638 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 128508us
I20260709 12:19:08.435492 24196 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:08.436583 24407 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.440852 24518 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:08.448537 24510 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:08.456765 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.484957 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:08.485918 18291 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:08.444095 24442 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.485114 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.455426 24133 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:08.493261 24133 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.436378 24449 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.457154 24563 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:08.436225 24410 consensus_queue.cc:1059] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:19:08.441387 24283 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:08.508612 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.464632 24529 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:08.516821 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 66795.2us
I20260709 12:19:08.483460 24405 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:08.496835 24133 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:08.516407 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104182us
I20260709 12:19:08.522835 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:08.506249 24407 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.458333 24308 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:19:08.534423 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:08.530691 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.536299 18366 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:08.496002 24520 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.514346 24442 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:08.544073 24442 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.459280 24159 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:08.548347 24159 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:19:08.551036 24159 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:08.551627 24159 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:08.552626 24159 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:08.560777 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 75700.6us
W20260709 12:19:08.516505 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 78116.8us
I20260709 12:19:08.519754 19007 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.528070 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.529366 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.569324 18375 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:08.532205 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.574433 18367 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:08.574816 18367 pending_rounds.cc:85] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 11
I20260709 12:19:08.575035 18367 pending_rounds.cc:107] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:08.575269 18367 raft_consensus.cc:2889] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.579432 19006 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.497216 24522 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.582917 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.533879 24529 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:08.607554 24567 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:19:08.599417 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 79716us
I20260709 12:19:08.576617 24322 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:08.578310 24541 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:08.577633 24552 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:08.631083 18375 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:08.628326 23946 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:08.646430 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:08.648514 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 66222.5us
I20260709 12:19:08.648069 18449 raft_consensus.cc:2368] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18.
I20260709 12:19:08.588328 24496 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:19:08.642910 24551 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.652705 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.628970 24577 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:08.668763 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.645290 24335 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:08.646835 24541 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.678894 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.680851 18369 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:19:08.693883 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.696084 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.698282 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:08.699002 18370 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:08.651628 24442 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.701651 24442 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:08.701689 24514 consensus_queue.cc:1059] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:08.702240 24442 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.703011 24514 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:08.703516 24514 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:08.703809 24579 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:08.704463 24514 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:08.705113 24570 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:19:08.706339 24522 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.706557 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:08.707213 24514 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.707683 18264 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.708535 24539 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:08.709360 18264 leader_election.cc:400] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:19:08.709700 18264 leader_election.cc:403] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:08.710034 24539 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.710481 24539 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:08.710852 24578 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:19:08.710882 24539 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.711108 24442 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.711809 24283 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.711037 24525 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:08.712436 24335 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14.
I20260709 12:19:08.711414 24535 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:19:08.716256 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.717939 24335 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.718467 24335 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.714040 24442 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.721416 24335 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:08.723899 24571 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:19:08.760018 24449 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:08.648036 24552 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:08.763898 24552 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.677951 24196 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.765708 24196 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:08.766582 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.766844 24196 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.772128 23884 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.672807 23946 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:08.784479 23946 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.644626 24563 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.706966 24541 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.676656 23714 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:08.795789 24510 raft_consensus.cc:3055] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:08.796211 24510 raft_consensus.cc:740] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:08.797008 24510 consensus_queue.cc:260] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.798173 24510 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.798975 19043 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:08.676746 24577 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.799832 24577 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:08.804673 18442 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.805177 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46956 (request call id 1421) took 1137 ms. Trace:
I20260709 12:19:08.805462 18442 rpcz_store.cc:276] 0709 12:19:07.667748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.668049 (+   301us) service_pool.cc:224] Handling call
0709 12:19:08.803623 (+1135574us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54003b1fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e73a3d 00007f3c6de43cd0 00007f3c673c8254 00007f3c673c81e2 00007f3c673c8111 00007f3c673c808c 00007f3c673c7342 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:08.805104 (+  1481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419305344}
I20260709 12:19:08.684209 18368 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
W20260709 12:19:08.835944 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 412171us
I20260709 12:19:08.823292 24011 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.758939 24429 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.836870 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.839200 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:08.843065 19043 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1237) took 1510 ms (client timeout 1731 ms). Trace:
I20260709 12:19:08.843438 19040 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:08.847342 19040 pending_rounds.cc:85] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 11
I20260709 12:19:08.847822 19040 pending_rounds.cc:107] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:08.848109 19040 raft_consensus.cc:2889] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:08.847286 19043 rpcz_store.cc:269] 0709 12:19:07.333277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.334209 (+   932us) service_pool.cc:224] Handling call
0709 12:19:08.842852 (+1508643us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400660154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:08.843006 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3365697280}
I20260709 12:19:08.860167 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:08.673832 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:08.861945 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 188386us
I20260709 12:19:08.863102 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:08.865089 24449 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:08.865967 24449 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:08.866288 24449 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:08.867069 24449 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.868304 24449 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:08.871400 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.872186 19044 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:08.872558 19044 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:08.862682 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1251) took 1391 ms. Trace:
I20260709 12:19:08.875010 19040 rpcz_store.cc:276] 0709 12:19:07.471529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.523548 (+ 52019us) service_pool.cc:224] Handling call
0709 12:19:08.843072 (+1319524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.848547 (+  5475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.862510 (+ 13963us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400660150. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:08.862630 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3033740160}
I20260709 12:19:08.878800 18338 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:08.870785 19035 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 8 to 10, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:08.887498 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.862987 24318 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:08.888350 19037 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.889101 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:08.863504 24504 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.874213 24202 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:08.843686 24011 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.639187 18193 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 7 to 8, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 8 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:08.897955 19044 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:08.884948 23714 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.895691 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:08.894153 24496 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:08.896117 24318 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:08.844165 24529 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:08.887145 23884 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:08.907316 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.884301 24595 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:08.914276 19041 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:08.863306 24436 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:19:08.897017 24504 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.923779 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:08.926484 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:08.903507 19044 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.927452 19037 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:08.945828 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:08.974337 18415 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.66:44577 timed out after 1.541s (SENT) [suppressed 40 similar messages]
W20260709 12:19:09.072764 18339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.008s	sys 0.004s
I20260709 12:19:09.001577 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:09.008134 18366 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:09.103672 18300 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.942942 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:09.021610 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 295) took 27545 ms (client timeout 15000 ms). Trace:
I20260709 12:19:09.002353 18299 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:19:08.913645 24531 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.001s
I20260709 12:19:08.935408 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:08.929111 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.945833 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:08.941036 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.023957 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:09.108088 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.946280 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:09.108686 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84855.1us
W20260709 12:19:09.103204 18339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.010s	sys 0.005s
I20260709 12:19:09.007341 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:09.109473 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 573739us
I20260709 12:19:09.109921 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:09.105306 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 198086us
I20260709 12:19:08.953061 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:09.111657 18372 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:09.125772 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:09.125804 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:09.123291 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:09.127986 18366 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:19:09.076262 24497 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.000s
W20260709 12:19:09.123953 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.913978 24594 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:19:09.123472 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 185544us
I20260709 12:19:09.143390 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:09.152391 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.935552 24318 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.007325 24222 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.984092 24335 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.923695 24283 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.159752 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50745.7us
I20260709 12:19:09.037678 24522 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.106352 24595 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.105540 24504 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.939826 24496 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:09.168001 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 285578us
I20260709 12:19:09.187129 19007 leader_election.cc:400] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18.
I20260709 12:19:09.173041 19044 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:09.180238 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:09.165876 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:08.902614 24415 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:09.129274 24599 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:19:09.120445 24585 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.923378 24159 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.182415 19041 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:09.184224 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:09.212898 19042 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:09.185374 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:09.121022 24196 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:09.163616 24594 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.168174 24598 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:09.178895 24514 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.176447 24342 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.227972 18447 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:09.231619 18444 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:09.188479 24595 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:09.189756 24335 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:09.247339 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 122202us
I20260709 12:19:09.187525 24522 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.218981 24519 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:09.257388 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 77267.8us
I20260709 12:19:09.227689 24436 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:09.189630 24222 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.191038 24283 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.267326 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:09.249493 24535 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:09.291456 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 165764us
I20260709 12:19:09.228127 24159 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.241191 24342 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.227149 24594 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:09.272320 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:09.231617 24514 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:09.277457 24410 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:09.275441 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 80900.5us
I20260709 12:19:09.294145 18368 raft_consensus.cc:2410] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:19:09.295298 18338 connection.cc:256] RPC call timeout handler was delayed by -1.4486374810001053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 12:19:09.302646 18338 connection.cc:256] RPC call timeout handler was delayed by -1.4321679360000417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 12:19:09.304173 18338 connection.cc:256] RPC call timeout handler was delayed by -1.3544692240000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 12:19:09.304275 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:09.272073 24609 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.304337 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:09.304798 18338 connection.cc:256] RPC call timeout handler was delayed by -1.203874869999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 12:19:09.305914 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:09.276043 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 301728us
W20260709 12:19:09.316876 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 105266us
W20260709 12:19:09.105274 18196 rpcz_store.cc:269] 0709 12:18:41.399336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.399625 (+   289us) service_pool.cc:224] Handling call
0709 12:19:08.940344 (+27540719us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:59.988455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:00.136461 (+148006us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:00.136511 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:19:00.136538 (+    27us) tablet.cc:662] Decoding operations
0709 12:19:00.137754 (+  1216us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:00.137851 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:00.137891 (+    40us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:19:00.138844 (+   953us) tablet.cc:701] Row locks acquired
0709 12:19:00.138869 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:19:00.139048 (+   179us) write_op.cc:270] Start()
0709 12:19:00.139215 (+   167us) write_op.cc:276] Timestamp: P: 1783599540138988 usec, L: 0
0709 12:19:00.157931 (+ 18716us) op_driver.cc:348] REPLICATION: starting
0709 12:19:00.207929 (+ 49998us) log.cc:844] Serialized 3616 byte log entry
0709 12:19:00.722577 (+514648us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c65d720ac 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b
0709 12:19:01.627278 (+904701us) op_driver.cc:464] REPLICATION: finished
0709 12:19:02.008509 (+381231us) write_op.cc:301] APPLY: starting
0709 12:19:02.008631 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:02.094172 (+ 85541us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:02.094232 (+    60us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:02.199211 (+104979us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:02.199501 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:19:02.199565 (+    64us) write_op.cc:312] APPLY: finished
0709 12:19:02.199949 (+   384us) log.cc:844] Serialized 89 byte log entry
0709 12:19:02.200121 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:02.200313 (+   192us) write_op.cc:454] Released schema lock
0709 12:19:02.200400 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:02.203791 (+  3391us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:02.203823 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":806384,"reported_tablets":47,"spinlock_wait_cycles":12032,"child_traces":[["op",{"apply.queue_time_us":365795,"apply.run_cpu_time_us":14514,"apply.run_wall_time_us":195300,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41711,"num_ops":9,"prepare.queue_time_us":626605,"prepare.run_cpu_time_us":43367,"prepare.run_wall_time_us":1244650,"raft.queue_time_us":381941,"raft.run_cpu_time_us":24046,"raft.run_wall_time_us":346269,"replication_time_us":1417367,"spinlock_wait_cycles":395827968,"thread_start_us":1964164,"threads_started":4,"wal-append.queue_time_us":955751,"wal-append.run_cpu_time_us":46961,"wal-append.run_wall_time_us":2025185}]]}
I20260709 12:19:09.277761 24519 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.324558 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:09.326465 18300 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:09.296196 24535 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:09.301594 24514 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.333110 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:09.332278 18415 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:09.334885 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:09.301874 24410 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.316402 24610 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:09.336230 24405 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:09.338797 24623 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:09.336403 24404 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:09.339419 24159 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:09.340054 24159 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:09.339807 24404 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.340467 24159 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:09.340952 24159 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:09.342238 24404 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:09.342872 24520 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:09.329706 24518 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.862s	user 0.006s	sys 0.006s
I20260709 12:19:09.283792 24496 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.322250 24609 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.358657 24405 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:09.343909 24410 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:09.381376 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.395457 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.363802 24405 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.364153 24610 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:09.416229 24610 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.417271 24335 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:09.360163 24539 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.000s
I20260709 12:19:09.347018 24623 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:09.357457 24518 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:09.360649 24612 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:19:09.392547 24404 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:09.435005 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 112174us
I20260709 12:19:09.436949 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:09.378654 24588 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:09.364703 24609 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:09.425987 24600 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:19:09.394604 24442 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:09.478024 18291 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:09.479080 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 171856us
I20260709 12:19:09.357299 24520 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.377715 24436 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 12:19:09.491012 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54157.4us
W20260709 12:19:09.495911 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 171483us
I20260709 12:19:09.496346 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:09.424072 24623 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:09.422838 24625 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:09.504303 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:09.511250 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 163672us
I20260709 12:19:09.518339 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:09.519569 19035 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:09.521636 18291 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:09.521761 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:09.522455 18443 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:09.436888 24539 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.487787 24588 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:09.504776 24415 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:09.433588 24504 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:09.527091 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 361304us
I20260709 12:19:09.510916 24609 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.473492 24535 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.604763 24631 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:09.520190 23946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.007s	sys 0.002s
I20260709 12:19:09.606619 24635 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:09.608017 24541 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.815s	user 0.011s	sys 0.000s
I20260709 12:19:09.608105 24635 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:09.608592 24510 consensus_meta.cc:287] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.002s
I20260709 12:19:09.613539 24595 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.614432 24514 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.615097 24539 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:09.587572 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:09.499778 24619 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:19:09.520793 24283 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.537233 24626 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.680605 18446 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:19:09.680646 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 91367.5us
I20260709 12:19:09.521267 24222 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.496910 24616 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 12:19:09.555456 24604 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:19:09.576212 24415 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:09.580160 24011 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:09.584630 24504 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:09.596593 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.567162 24552 consensus_meta.cc:287] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.009s
I20260709 12:19:09.512248 24522 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:09.681968 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:09.685500 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52050.1us
I20260709 12:19:09.695730 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:09.695999 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:09.696297 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:09.696578 18368 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.698397 18445 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:09.713721 18368 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:09.677386 24525 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:09.676905 24609 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:09.717324 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 351749us
I20260709 12:19:09.680269 24626 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.736449 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:09.688428 24519 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:09.737133 18442 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.714841 18368 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.743925 18368 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.608618 24541 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.703751 24283 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.737196 24609 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.758113 18368 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:09.718695 24525 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.740868 24626 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.705070 24605 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:19:09.769404 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:09.708673 24436 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.703640 24415 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:09.768772 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 453018us
I20260709 12:19:09.785102 18374 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:09.723696 24603 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:19:09.763520 24335 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.764660 24598 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:09.709311 24552 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.780946 24525 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.773711 24539 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.792850 24436 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:09.803946 24640 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:09.809331 18193 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:09.780114 24283 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.811856 24514 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.812919 19043 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:09.813549 24283 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.813948 24552 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:09.794243 24405 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:09.814318 24335 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.717545 24588 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.757026 24618 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:19:09.786679 24595 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.815132 24405 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.819104 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:09.820995 24539 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.827564 24539 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:09.778873 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:09.789642 18442 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:09.828039 24335 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.833844 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:09.809535 24159 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:09.816597 24449 consensus_meta.cc:287] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.948s	user 0.010s	sys 0.000s
W20260709 12:19:09.811875 24133 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.274s	user 0.011s	sys 0.001s
I20260709 12:19:09.861783 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:09.780491 24594 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.831831 24283 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:09.848814 24407 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:09.864354 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:09.866247 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:09.823132 24405 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.868108 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:09.912204 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44182.3us
W20260709 12:19:09.912572 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.834483 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:09.870090 24552 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.820853 24577 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.021s	user 0.010s	sys 0.000s
I20260709 12:19:09.888947 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:09.834221 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.012076 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:09.848387 24653 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:09.872983 24318 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.001s
I20260709 12:19:09.843020 24436 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:09.909336 18373 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:19:09.871546 24590 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:10.006206 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.025460 18373 pending_rounds.cc:85] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 12
I20260709 12:19:09.909971 24635 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:09.818568 24623 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.797489 24598 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:09.910283 24551 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:10.110481 24551 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.111877 24598 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.112906 24551 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.025913 18373 pending_rounds.cc:107] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:19:10.113482 18373 raft_consensus.cc:2889] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.909621 24407 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:10.115176 24407 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.116353 24407 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.997032 24449 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.033352 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:09.943569 24415 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.038003 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:10.135782 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 102621us
I20260709 12:19:09.942895 24283 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:09.833544 18193 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 11, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:10.026346 24577 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.020082 24552 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.046053 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.049204 24655 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:10.051954 24641 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:19:10.072582 24594 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.003789 24541 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.080737 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.138577 24436 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:10.003139 24595 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:19:10.085655 24335 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:09.889899 24133 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.028410 24653 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.028052 24318 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:10.098598 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 78712.5us
I20260709 12:19:10.099470 18368 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:10.099639 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.101756 24635 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:10.102502 24539 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.103780 24342 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.107276 24590 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.109669 24623 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.118407 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.127131 18448 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:10.139753 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:10.122077 24496 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:10.082739 24159 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.174793 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.147730 24525 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.181088 19035 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: []
W20260709 12:19:10.191223 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 140626us
I20260709 12:19:10.187810 18370 raft_consensus.cc:2368] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:10.190111 19043 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
W20260709 12:19:10.190124 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50497.1us
I20260709 12:19:10.192838 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:10.194341 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:10.195099 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:10.195003 18441 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.196429 19041 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:10.199573 18291 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:10.201365 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:10.202369 18448 raft_consensus.cc:2410] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:19:10.203068 18444 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:10.153853 24655 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.204030 18448 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:10.208546 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:10.209125 18297 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.211930 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.213702 24665 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:10.213900 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.214645 18294 raft_consensus.cc:2393] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:10.214414 24665 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.215369 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:10.215924 18294 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.217128 24665 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.217788 24665 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:10.218252 18297 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:10.218400 24665 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.220157 18295 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:10.190940 24608 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.220675 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.199023 24159 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:10.193542 24535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.022s	sys 0.009s
I20260709 12:19:10.191634 24342 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:10.202482 24410 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.000s
I20260709 12:19:10.225595 18448 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 12:19:10.230484 18294 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:10.231710 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.242969 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.277647 18366 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:10.278540 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.243712 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.279836 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.246052 18441 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:10.281293 18441 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:10.251616 18371 raft_consensus.cc:3055] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:10.282970 18371 raft_consensus.cc:740] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:10.197924 24623 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.266711 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.185204 24539 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.211642 24449 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:10.273312 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.290930 18295 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 11.
I20260709 12:19:10.198508 24609 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.297984 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.299052 18296 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:10.303918 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.275542 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:10.289444 18370 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:19:10.270087 24535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.022s	sys 0.009s
I20260709 12:19:10.250654 24618 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.219682 18293 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 8 to 10, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:10.194612 24496 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.250525 24563 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.294554 18448 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:10.283886 18371 consensus_queue.cc:260] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.307888 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.346678 18371 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:10.329169 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53694.7us
I20260709 12:19:10.223500 24595 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:19:10.349750 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.272698 24608 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:10.333518 24654 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:19:10.334374 24283 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.335333 24529 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:10.298019 24623 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.337349 24618 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.232561 24594 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:19:10.342628 18447 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:10.343681 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.356617 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2658) took 1431 ms. Trace:
I20260709 12:19:10.357270 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.365000 18369 raft_consensus.cc:2368] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:19:10.389026 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.364784 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.366703 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.390385 18370 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.288102 24410 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.371407 24608 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.377041 24496 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.295773 24449 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:10.380143 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.340189 24415 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:10.234465 24635 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.369453 24648 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:10.388376 18449 rpcz_store.cc:276] 0709 12:19:08.925137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.927051 (+  1914us) service_pool.cc:224] Handling call
0709 12:19:08.927427 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.948031 (+ 20604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.948074 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.956587 (+  8513us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.209013 (+252426us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:09.251043 (+ 42030us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.263050 (+ 12007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.263259 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.354107 (+1090848us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b1800156820. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:19:10.356351 (+  2244us) raft_consensus.cc:1685] finished
0709 12:19:10.356440 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.356564 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20140,"prepare.queue_time_us":61196,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":32596,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":41399,"spinlock_wait_cycles":10570752,"thread_start_us":221281,"threads_started":1,"wal-append.queue_time_us":338820}
I20260709 12:19:10.416155 18449 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:10.391983 18371 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:19:10.394249 18448 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:10.397464 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.327526 24629 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:19:10.401902 18366 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:19:10.427970 18366 pending_rounds.cc:85] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 11
I20260709 12:19:10.428236 18366 pending_rounds.cc:107] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:19:10.428472 18366 raft_consensus.cc:2889] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.407244 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:10.429801 18447 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:10.414503 18370 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:19:10.416626 18371 pending_rounds.cc:85] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 8
I20260709 12:19:10.432070 18371 pending_rounds.cc:107] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:19:10.432329 18371 raft_consensus.cc:2889] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.436182 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.364466 24563 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.392362 24552 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.437955 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.440416 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.402727 24449 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:10.405540 24577 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.407509 24608 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:10.451876 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.431301 24415 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:10.457477 24525 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.440862 24318 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:10.458433 24662 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:19:10.464406 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.464578 24655 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.469254 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:10.449920 24552 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:10.487687 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:10.488261 18296 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:10.489405 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.469612 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.474052 24429 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.474661 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.475057 24618 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.475374 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.491698 18295 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:10.492723 18295 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:10.391929 24529 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.437747 24133 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:10.452323 24656 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
W20260709 12:19:10.459106 24660 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:10.490939 18294 raft_consensus.cc:2410] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:19:10.494216 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 130229us
I20260709 12:19:10.496196 19009 leader_election.cc:400] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:10.496130 18295 raft_consensus.cc:1240] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:19:10.496206 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.496631 19009 leader_election.cc:403] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:10.498569 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.499598 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.506107 18296 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:10.500185 18366 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:10.516587 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:10.523980 24633 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.942s	user 0.003s	sys 0.000s
W20260709 12:19:10.526105 24657 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:19:10.527932 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.476991 24608 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.531081 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1402) took 1881 ms. Trace:
I20260709 12:19:10.531575 18373 rpcz_store.cc:276] 0709 12:19:08.650015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.650182 (+   167us) service_pool.cc:224] Handling call
0709 12:19:08.922356 (+272174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.113963 (+1191607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.530779 (+416816us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540030ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:10.531007 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189397760}
I20260709 12:19:10.536543 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2656) took 1612 ms. Trace:
I20260709 12:19:10.537698 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.538309 19041 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:10.539101 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.539338 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.540382 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.542014 18373 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:10.542290 19043 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:10.543372 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.485491 24590 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.536908 18440 rpcz_store.cc:276] 0709 12:19:08.924628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.924739 (+   111us) service_pool.cc:224] Handling call
0709 12:19:08.925013 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.949082 (+ 24069us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:09.197238 (+248156us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.480952 (+283714us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.481423 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.481550 (+   127us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:09.481581 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.481643 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:09.815737 (+334094us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b48001e0258. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6dda5aa1 00007f3c6dd9cb37 00007f3c6dd9c9ca 00007f3c6de9426f 00007f3c6de93f37 00007f3c6dea855c 00007f3c6dea8508 00007f3c6dea8488 00007f3c6dea8443 00007f3c6dea76ec 00007f3c6cdd1a96
0709 12:19:10.523439 (+707702us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:10.529377 (+  5938us) raft_consensus.cc:1685] finished
0709 12:19:10.529398 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.536488 (+  7090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24224,"prepare.queue_time_us":37831,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":23935,"raft.run_cpu_time_us":3465,"raft.run_wall_time_us":868070,"spinlock_wait_cycles":76691328,"thread_start_us":493275,"threads_started":2,"wal-append.queue_time_us":315260}
I20260709 12:19:10.545768 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.551860 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:10.552973 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:10.556137 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.557726 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.558418 18369 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.557065 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.561453 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.518528 24627 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:10.565042 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.565119 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.572204 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.572659 19044 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.566382 18367 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:19:10.567466 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:10.572781 18372 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:10.577886 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:10.566196 24626 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.003s
I20260709 12:19:10.496425 24552 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.585876 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:10.587117 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.587431 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.587759 19037 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
W20260709 12:19:10.588519 24673 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
I20260709 12:19:10.594045 24645 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:10.594596 24645 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.572664 24590 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:10.565308 24665 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.578572 24322 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:10.610024 19044 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:10.578480 24653 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.610592 24496 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:10.611100 24623 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.608635 24618 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.612324 24563 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.583215 24415 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.578707 24539 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.586521 24609 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.614524 24410 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.577742 24497 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:19:10.578761 24655 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:19:10.585731 24619 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:19:10.623698 24529 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.635849 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.597169 24674 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:10.642092 24685 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.618096 24322 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:10.617147 24590 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:10.602097 24626 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.644349 24645 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.650072 24655 consensus_queue.cc:1243] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 3da7ad8fa20a4591963b660e4c848e7f log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:10.652202 24674 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.653842 24322 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.653347 24552 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.668275 24683 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.657161 24685 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.648761 24449 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.756367 24520 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.758525 24449 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.761358 24552 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:10.714056 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 139250us
I20260709 12:19:10.720106 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:10.607352 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.766274 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:10.724773 18368 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:10.728708 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.769171 24623 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.731165 18375 raft_consensus.cc:1240] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:10.732717 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2898) took 1031 ms. Trace:
I20260709 12:19:10.732779 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.740177 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:10.744190 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.745500 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.745543 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.677656 24590 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:10.792037 19042 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:10.676864 24626 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.675065 24627 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
W20260709 12:19:10.681569 24636 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.109s	user 0.002s	sys 0.000s
I20260709 12:19:10.670480 24674 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.672564 24683 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.774617 19039 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: [16.14-17.15]
I20260709 12:19:10.775897 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:10.781024 18373 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:10.781785 24675 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.792580 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:10.802778 24685 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.812026 24552 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:10.813529 24675 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.817739 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:10.816263 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2353) took 1864 ms. Trace:
W20260709 12:19:10.826303 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 106635us
I20260709 12:19:10.819052 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:10.835359 18369 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:10.836684 24659 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:10.837208 24659 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:10.837493 24627 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.838603 24594 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:10.839207 24594 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:10.840900 24594 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:10.842976 24678 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:19:10.828037 18298 rpcz_store.cc:276] 0709 12:19:08.952053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.971756 (+ 19703us) service_pool.cc:224] Handling call
0709 12:19:08.972240 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:08.984396 (+ 12156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.984440 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:09.022845 (+ 38405us) log.cc:844] Serialized 52 byte log entry
0709 12:19:09.474883 (+452038us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:09.689482 (+214599us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.692184 (+  2702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.692254 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.680950 (+988696us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:10.815615 (+134665us) raft_consensus.cc:1685] finished
0709 12:19:10.815753 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.816036 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20288,"prepare.queue_time_us":313838,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":33193,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":10874,"spinlock_wait_cycles":26029952,"thread_start_us":653210,"threads_started":2,"wal-append.queue_time_us":485578}
I20260709 12:19:10.845917 24659 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.847620 24335 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:10.849864 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:10.851409 24626 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:10.853454 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2647) took 1158 ms. Trace:
I20260709 12:19:10.854001 18375 rpcz_store.cc:276] 0709 12:19:09.694787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.694940 (+   153us) service_pool.cc:224] Handling call
0709 12:19:09.695192 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.770199 (+1075007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.853034 (+ 82835us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003f1954. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:10.853357 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380887296}
I20260709 12:19:10.829733 24449 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:10.858407 24449 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.862227 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.815171 24619 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:10.866659 24619 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:10.867453 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.867995 24619 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.869582 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1257) took 1004 ms. Trace:
I20260709 12:19:10.869715 24619 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.870016 19035 rpcz_store.cc:276] 0709 12:19:09.865240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.865416 (+   176us) service_pool.cc:224] Handling call
0709 12:19:09.865759 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.191566 (+325807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.869068 (+677502us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540048ec54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:10.869376 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724525824}
I20260709 12:19:10.871445 24539 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.866257 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1392) took 2340 ms. Trace:
I20260709 12:19:10.770963 18444 rpcz_store.cc:276] 0709 12:19:09.701118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.203932 (+502814us) service_pool.cc:224] Handling call
0709 12:19:10.204199 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.204559 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:10.204583 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.204989 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.599173 (+394184us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b1800424c00. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:19:10.731819 (+132646us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.731911 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.732025 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.732083 (+    58us) raft_consensus.cc:1685] finished
0709 12:19:10.732096 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.732410 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18493,"raft.queue_time_us":98,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":13794,"spinlock_wait_cycles":18536192,"thread_start_us":132048,"threads_started":1,"wal-append.queue_time_us":470737}
I20260709 12:19:10.872102 18374 rpcz_store.cc:276] 0709 12:19:08.526203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.526338 (+   135us) service_pool.cc:224] Handling call
0709 12:19:08.526719 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.787531 (+1260812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.865718 (+1078187us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000f8554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:10.866084 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2893276544}
I20260709 12:19:10.873268 24608 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.874874 24552 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:10.876861 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.877911 18371 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:10.880155 24675 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.881889 24683 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.883245 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.887750 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.889151 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.889381 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:10.889848 18368 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:10.893101 24603 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:10.854031 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:10.895254 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 155417us
I20260709 12:19:10.895619 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:10.906754 18372 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:10.907407 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 88560.8us
I20260709 12:19:10.819907 24322 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:10.934677 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52958.8us
I20260709 12:19:10.954645 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:10.958774 19041 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:10.963179 24449 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:10.968688 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:10.961962 24552 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:10.972422 24322 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:10.980242 24606 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:19:10.980813 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:10.982092 24616 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:10.982848 24405 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:10.983636 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 129817us
I20260709 12:19:10.977257 24539 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:10.989382 24322 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:10.993148 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:10.996450 18448 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:11.010737 24626 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:11.011050 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:11.013345 24645 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:11.013839 24645 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:11.014305 24645 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.019762 24529 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:11.020277 24529 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:11.020805 24529 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.023245 24659 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.000710 24449 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:11.027782 24683 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:11.028260 24683 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:11.028358 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.029383 24683 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.034049 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:11.035651 24690 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.377s	user 0.002s	sys 0.001s
I20260709 12:19:11.048636 24539 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:11.049875 24539 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:11.050335 24539 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:11.051885 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:11.052443 24598 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.052789 18299 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:11.053043 24598 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.031005 24438 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.056532 24438 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.057720 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.059212 24438 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.060443 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.060725 24497 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:11.061084 18292 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:11.055351 24405 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:11.064401 24630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.389s	user 0.000s	sys 0.021s
W20260709 12:19:11.064862 24630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.493s	user 0.000s	sys 0.022s
I20260709 12:19:11.065524 24405 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:11.066817 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118453us
I20260709 12:19:11.068773 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.068943 24322 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.059319 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.085700 24497 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:11.074831 24598 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.086023 24605 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.086697 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.087607 24609 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.088449 24609 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:11.089128 24609 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.092024 24449 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:11.095422 24626 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:11.095808 24626 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.096815 24626 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.098181 24626 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:11.101553 24322 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.079404 24683 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.090478 24609 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.112176 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:11.102242 24539 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.120695 18299 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:11.121377 24539 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:11.124037 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 55381.6us
I20260709 12:19:11.130402 18448 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:11.131654 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:11.132189 24405 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.132617 24322 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.132941 24405 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.133373 24322 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.135632 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.135159 24405 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:11.136831 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.136008 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.137434 19040 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:11.135771 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:11.139647 24628 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.140100 24674 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.140305 24628 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.141196 24602 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.141762 24602 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.142231 24628 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.129336 24630 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:11.149051 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:11.150782 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90551.7us
I20260709 12:19:11.152063 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:11.156167 18300 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:11.142268 24674 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:11.156260 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:11.157196 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.147959 24529 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.163441 24659 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:11.163772 18372 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.164237 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.165133 18441 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:11.163445 24531 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:19:11.167410 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.157943 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.170086 18366 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:11.171648 18264 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.172567 24598 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.173261 24598 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.173744 24598 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.175559 24631 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:11.175807 24011 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:11.177585 24629 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:19:11.182180 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:11.145103 24630 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.185734 18339 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.176901 24598 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.187635 18372 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:11.163352 24529 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.180094 24659 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:11.194160 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.158270 24322 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.197587 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:11.197712 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.198006 24449 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:11.198755 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 55421.2us
I20260709 12:19:11.203068 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.198880 24588 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:11.199134 18296 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:11.204413 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:11.207114 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.205374 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.208400 18266 leader_election.cc:400] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:19:11.213223 18266 leader_election.cc:403] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:11.213261 18372 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:11.197791 24011 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:11.215392 19035 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:11.217897 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.218724 18291 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:11.204728 24588 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:11.220817 19039 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:11.221076 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.221691 18446 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:11.219997 19036 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:11.228531 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.204056 24659 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.230142 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.230541 19042 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:11.214754 24616 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:11.233675 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.230104 18266 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.233534 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:11.235109 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.235271 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:11.236364 24514 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:11.238076 24514 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:11.238451 24514 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.238511 24535 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:11.238938 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.239935 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.241037 24449 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.241832 18339 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.241995 24449 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:11.242337 24626 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:11.242861 24644 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:19:11.222550 24531 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.243937 24202 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:11.244498 24202 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:11.244644 24590 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.244405 24531 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.245656 24202 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.247037 24531 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.249595 24590 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:11.252266 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.252936 24630 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.253917 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.254694 24630 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.239952 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.258188 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.258926 19035 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:11.238914 24514 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.243477 24588 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.266239 24335 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.266760 24588 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.267083 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:11.269526 24593 consensus_queue.cc:1175] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3da7ad8fa20a4591963b660e4c848e7f" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 14 index: 11 } }
I20260709 12:19:11.267088 18293 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:11.272322 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.241091 24598 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:11.333284 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:11.312167 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 70403.3us
I20260709 12:19:11.266791 24335 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.344923 24429 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:11.345767 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 113038us
I20260709 12:19:11.346256 24674 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:11.349169 24674 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:11.349607 24449 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:11.350175 24449 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.346145 24405 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.350209 24674 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.351503 24449 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.352370 24283 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:11.353010 24449 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.354096 24283 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:11.354539 24283 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.355830 24283 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.357105 24707 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:11.357234 24283 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.357812 24707 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.359488 24645 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.358207 24659 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.343655 24598 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:11.352432 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:11.364085 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.374964 24645 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.375147 24335 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.376458 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.377664 24525 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 14. Current term is 15.
I20260709 12:19:11.378809 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.379575 18440 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:19:11.380982 24710 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:19:11.382002 24616 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:11.382500 24616 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.383826 24616 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.385485 24616 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.386497 24514 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.387360 24429 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:11.388015 18291 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:11.333168 24594 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:11.346818 24685 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:11.388456 24598 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:11.408617 24659 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:11.414786 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62521.7us
W20260709 12:19:11.415099 18183 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:11.418210 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.419030 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:11.394290 24593 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:11.399158 24674 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:11.423581 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:11.411746 24594 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.396560 24685 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:11.425418 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 151534us
I20260709 12:19:11.426299 24685 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.427901 24685 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:11.435657 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 181826us
I20260709 12:19:11.437776 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.440757 24593 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.442152 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.442760 24602 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.443807 24659 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:11.444586 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.404350 24626 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.444911 24659 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.445405 24629 consensus_queue.cc:1059] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:11.446218 24629 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:11.447398 24335 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:11.448124 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.446981 24629 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.425428 24594 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.447433 24535 raft_consensus.cc:3055] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:11.466648 24645 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:11.466815 24407 consensus_queue.cc:1059] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:11.409714 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.463522 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.473280 18372 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:11.475133 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.393911 24011 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 12:19:11.477798 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:11.470247 24335 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.495563 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.475843 24535 raft_consensus.cc:740] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:11.503556 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.476854 24602 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.497664 18369 pending_rounds.cc:85] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 6
I20260709 12:19:11.514073 24514 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.517083 24429 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:11.518267 24407 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:11.518874 24407 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.519929 24407 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.522081 24645 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.522485 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.522796 24645 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:11.523416 18441 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:11.523321 24645 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.523991 24707 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.524452 24645 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.525856 24645 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.513131 18369 pending_rounds.cc:107] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:11.526647 18369 raft_consensus.cc:2889] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:11.527565 24497 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:11.527997 24497 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:11.529544 24496 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.530172 24496 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.532553 24496 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.527568 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:11.533932 19036 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:11.535851 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64852.2us
I20260709 12:19:11.507643 24629 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.541785 24629 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.545591 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.546291 19042 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:11.548152 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.495594 24335 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:11.564086 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69353.3us
I20260709 12:19:11.564718 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:11.567422 18266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.578593 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.549247 24514 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:11.558120 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.589152 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:11.553993 19036 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:11.523156 24405 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:11.584664 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:11.595656 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 193533us
I20260709 12:19:11.599681 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:11.603066 18448 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:11.607877 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:11.609721 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:11.603642 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:11.611440 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72190.2us
I20260709 12:19:11.612258 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.613013 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.614324 24496 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.615084 24497 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:11.616382 24497 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.619282 24610 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:11.619482 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.619776 24610 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:11.620252 24610 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.620589 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.621685 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.622537 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.623593 24075 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:11.623503 24011 consensus_queue.cc:1059] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:11.624722 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.629223 18298 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:19:11.589358 24514 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.631145 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:11.606359 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:11.629706 18298 pending_rounds.cc:85] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 14
I20260709 12:19:11.640105 18298 pending_rounds.cc:107] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:19:11.640434 18298 raft_consensus.cc:2889] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:11.643939 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 47059.4us
I20260709 12:19:11.566668 24602 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.656721 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.657526 18444 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:11.660521 18263 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:11.646107 24514 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.690220 19040 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:11.690151 18291 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.684894 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:11.691780 18448 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.695456 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.662497 24496 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:11.663157 24529 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:11.665290 24593 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:11.748709 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.719569 24518 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.720124 24635 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:11.721114 24626 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:11.721721 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 67279.2us
I20260709 12:19:11.674749 24535 consensus_queue.cc:260] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:11.733174 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 110712us
W20260709 12:19:11.734309 24714 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:11.735587 24602 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:11.739380 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:11.712926 24496 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:11.770088 24496 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.770838 24518 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:11.771831 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.772241 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.772701 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.773067 24518 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.778818 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.781785 24707 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:11.782379 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.820479 18366 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:11.820911 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:11.758566 18291 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:11.662374 24202 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:19:11.782455 24707 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.768400 24635 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:11.828539 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 222131us
I20260709 12:19:11.861034 18298 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:19:11.864995 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.785015 24598 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.883790 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:11.827430 24535 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.796643 24529 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.830482 24609 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:19:11.871881 24659 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:11.878365 24283 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:11.900669 18369 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:11.901172 24529 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.901693 18373 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:11.929941 18369 pending_rounds.cc:85] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 9
I20260709 12:19:11.904512 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:11.905077 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.930222 18369 pending_rounds.cc:107] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:11.905400 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:11.910104 24598 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.911926 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:11.903036 24707 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:11.913404 18449 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:11.919539 24635 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:11.920377 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:11.923336 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.929124 18370 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:11.903106 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:11.930568 18369 raft_consensus.cc:2889] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:11.939862 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:11.926589 24449 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 12:19:11.950285 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:11.940299 24202 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:11.942147 24707 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.955657 24322 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:11.938519 24283 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:11.938907 24659 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:11.994998 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 347097us
W20260709 12:19:11.967859 24616 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.008s
I20260709 12:19:11.965615 24529 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:11.965801 18368 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 8 to 10, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:11.964640 24449 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:19:11.945078 24628 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:19:11.998493 18373 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:19:12.014739 24719 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 12:19:11.999799 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 94499.8us
I20260709 12:19:12.020187 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:12.018194 19036 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:12.022855 18264 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:12.032825 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.010459 24539 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:12.040410 18291 raft_consensus.cc:1240] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:12.013381 24529 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:12.049239 24645 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.002s
I20260709 12:19:12.049921 18367 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:12.052474 18375 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:12.055770 24605 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:12.058182 24626 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:12.061501 24616 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:12.062208 24616 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.064785 24616 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.065876 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.071481 24610 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.074082 24610 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.034888 24525 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.084653 24659 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.101056 19007 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:12.113730 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 325840us
I20260709 12:19:12.114557 18448 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
W20260709 12:19:12.100147 24719 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:19:12.097549 24685 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
I20260709 12:19:12.097332 24630 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:12.099926 24683 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:12.098007 24529 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:12.130481 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 107715us
I20260709 12:19:12.137668 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:12.144896 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 113094us
I20260709 12:19:12.124670 24674 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.087196 24075 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:19:12.176115 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44292.8us
I20260709 12:19:12.178766 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.179730 18291 raft_consensus.cc:2393] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:12.137050 24497 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.096241 24539 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.094688 24496 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:12.128917 24335 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 12:19:12.202059 19007 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:12.271394 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 70356us
I20260709 12:19:12.146160 24683 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:12.273576 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 71605.4us
I20260709 12:19:12.274442 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.145255 24630 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:12.288388 18369 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:12.297170 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:12.179001 24075 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:12.329499 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.361461 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 76988.7us
I20260709 12:19:12.180945 24616 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.107s
W20260709 12:19:12.377859 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 81155.1us
I20260709 12:19:12.200469 24497 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.296979 24725 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.336256 24602 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.336539 24631 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:12.406247 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:12.406455 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:12.401809 24535 consensus_meta.cc:287] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.510s	user 0.005s	sys 0.006s
I20260709 12:19:12.256320 24335 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.407575 18291 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:12.407060 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:12.380429 24674 raft_consensus.cc:2764] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:12.426535 18293 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:12.412601 24725 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.433044 24725 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.416483 24631 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.436177 24631 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.436782 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.421595 24732 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:12.437740 24732 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.443949 18291 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:12.446465 24729 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:19:12.446914 19007 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:12.447567 24728 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.448181 24728 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.421730 24630 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.449361 24632 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.449780 24632 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:12.450104 24732 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.450387 24728 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:12.418542 24685 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:12.454425 24407 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.017s	sys 0.048s
W20260709 12:19:12.456571 18446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.026s	sys 0.058s
I20260709 12:19:12.420318 24628 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.450208 24632 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:12.459964 18446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.027s	sys 0.058s
W20260709 12:19:12.465629 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58695.1us
I20260709 12:19:12.429543 24626 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:12.464620 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.458636 24514 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.003s
I20260709 12:19:12.462651 24075 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.494163 24628 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.453832 24683 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.495420 24496 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.495636 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:12.496073 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.496775 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.499966 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:12.501281 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.555364 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.005s	sys 0.009s
W20260709 12:19:12.562642 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.005s	sys 0.009s
I20260709 12:19:12.462772 24497 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:12.549445 24588 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.047s	sys 0.046s
I20260709 12:19:12.569314 18366 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:12.466079 24602 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:12.557731 24635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.005s	sys 0.008s
I20260709 12:19:12.585577 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:12.587423 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.592873 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.592901 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.593493 18366 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:12.600085 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:12.600558 24734 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:19:12.601441 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.601949 24644 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.602452 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1100) took 1093 ms. Trace:
I20260709 12:19:12.602813 18299 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:12.603358 24075 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:12.604084 18299 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:12.605096 24429 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:12.607864 18296 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:19:12.611369 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.607050 24429 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:19:12.612208 24726 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:12.612818 24726 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.602483 24644 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.619105 24644 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.619633 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.620277 18294 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:12.620424 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.621071 18366 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:12.622818 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:12.624208 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:12.624774 19043 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:12.625811 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:12.626627 24621 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.627174 24621 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.631098 24621 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:12.555241 24407 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 12:19:12.634032 18366 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:12.580711 24588 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.047s	sys 0.046s
I20260709 12:19:12.638144 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:12.614990 24726 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.632019 24621 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:12.650112 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 154595us
I20260709 12:19:12.554939 24514 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.679056 24514 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.647552 24407 consensus_queue.cc:1243] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:19:12.684201 18370 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:12.684649 18370 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:12.685515 18370 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.655802 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:12.648808 18414 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.008s	sys 0.057s
I20260709 12:19:12.649396 24707 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.634801 24539 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.659757 24609 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.663051 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2676) took 1048 ms. Trace:
I20260709 12:19:12.663172 24335 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:12.672187 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.672433 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.673333 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.673710 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.597062 24610 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:19:12.743317 18300 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:12.743525 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.744316 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.744511 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:12.745018 18300 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:12.745504 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.745725 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.746884 18300 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:12.747556 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.748879 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.749488 18448 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:12.750308 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:12.750996 18448 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:12.752629 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.754283 18449 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:12.755443 18444 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:12.754673 24632 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.593678 24602 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.758055 24602 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:12.594094 24724 consensus_queue.cc:1059] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:19:12.676597 24685 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:12.649140 24621 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.759140 24724 raft_consensus.cc:3055] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:12.759415 24685 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:12.687354 18442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.002s	sys 0.030s
W20260709 12:19:12.760172 18442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.002s	sys 0.030s
W20260709 12:19:12.688701 18414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.008s	sys 0.057s
I20260709 12:19:12.760532 24737 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:12.761093 24737 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.602856 19038 rpcz_store.cc:276] 0709 12:19:11.509006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.515292 (+  6286us) service_pool.cc:224] Handling call
0709 12:19:11.559872 (+ 44580us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:11.577413 (+ 17541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.577448 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:11.578022 (+   574us) log.cc:844] Serialized 52 byte log entry
0709 12:19:11.782485 (+204463us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b1800073840. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:19:12.018371 (+235886us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:12.127355 (+108984us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.174502 (+ 47147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.174602 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.602131 (+427529us) raft_consensus.cc:1685] finished
0709 12:19:12.602236 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.602378 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24193,"prepare.queue_time_us":107360,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":47187,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":20473,"spinlock_wait_cycles":5360000,"thread_start_us":235722,"threads_started":1,"wal-append.queue_time_us":776805}
I20260709 12:19:12.686968 18370 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:12.697446 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48938.3us
W20260709 12:19:12.729146 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.728679 18372 rpcz_store.cc:276] 0709 12:19:11.614448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.624880 (+ 10432us) service_pool.cc:224] Handling call
0709 12:19:11.663735 (+ 38855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.754255 (+ 90520us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:11.754284 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:11.780148 (+ 25864us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.268548 (+488400us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:12.268588 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.268655 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.662859 (+394204us) raft_consensus.cc:1685] finished
0709 12:19:12.662878 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.662989 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22950,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":488012,"threads_started":1,"wal-append.queue_time_us":879267}
W20260709 12:19:12.591373 24635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.006s	sys 0.009s
W20260709 12:19:12.690263 24593 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.016s	sys 0.051s
W20260709 12:19:12.737687 18183 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:12.738440 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.738762 24728 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:12.739459 18445 raft_consensus.cc:1240] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:12.740499 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:12.795637 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.759361 24621 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.593603 24616 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.759603 24724 raft_consensus.cc:740] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:12.760054 24685 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:12.800155 18368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.007s	sys 0.023s
I20260709 12:19:12.734506 18299 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:12.690107 24707 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.794667 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:12.740773 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.763101 24737 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:12.798646 18264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.022s
I20260709 12:19:12.798781 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:12.797746 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.732939 18293 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:12.803483 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.797158 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:12.802263 18368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.007s	sys 0.023s
W20260709 12:19:12.803432 18264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.008s	sys 0.022s
W20260709 12:19:12.793771 24593 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.016s	sys 0.051s
I20260709 12:19:12.789849 24514 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:12.816273 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.822155 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:12.822822 19038 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:12.820444 18367 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:12.826550 18300 raft_consensus.cc:1240] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 12:19:12.797960 18183 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 60348.1us
I20260709 12:19:12.812714 24707 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.826131 24644 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:12.843928 24644 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:12.850152 24529 consensus_meta.cc:287] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.564s	user 0.005s	sys 0.003s
I20260709 12:19:12.851660 24685 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.735756 24609 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.854954 24609 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.855786 24628 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.857731 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.858582 18294 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:12.859447 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:12.860065 18294 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:12.861274 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.850646 24529 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:12.862699 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.863559 24610 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.863705 18372 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:12.866377 24683 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.866298 24724 consensus_queue.cc:260] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.826851 24514 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.824206 24335 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:12.862046 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.902911 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:12.909010 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.909672 24630 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.909711 18441 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:12.910964 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:12.902873 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:12.912701 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.872665 18370 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:19:12.913350 18370 pending_rounds.cc:85] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 13
I20260709 12:19:12.914129 18370 pending_rounds.cc:107] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:19:12.914613 18370 raft_consensus.cc:2889] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.886281 24610 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.887571 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:12.916929 18440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.017s	sys 0.030s
I20260709 12:19:12.887961 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:12.917289 18440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.017s	sys 0.030s
I20260709 12:19:12.799436 24728 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:12.888633 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2674) took 1286 ms. Trace:
I20260709 12:19:12.917754 18375 rpcz_store.cc:276] 0709 12:19:11.593983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.594440 (+   457us) service_pool.cc:224] Handling call
0709 12:19:12.880569 (+1286129us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400394754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:12.880700 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982270208}
I20260709 12:19:12.888842 24674 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:12.891880 18297 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:12.892423 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.892948 24529 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.894809 24283 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:12.896579 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.896361 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:12.847034 24644 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.905465 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:12.907425 24514 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:12.907938 24335 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.943979 24335 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.913677 18372 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:12.885582 24632 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:12.879555 18266 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:12.908113 24707 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:12.948027 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72806.1us
I20260709 12:19:12.948495 24222 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:12.948968 24222 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:12.941885 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45611us
I20260709 12:19:12.948700 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:12.949380 24222 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.919303 18373 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:12.950126 18444 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.950570 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:12.952268 24514 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:12.952651 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:12.949940 24525 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:12.959638 19007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.025s
W20260709 12:19:12.960197 19007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.000s	sys 0.025s
W20260709 12:19:12.960697 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 512831us
I20260709 12:19:12.961269 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:12.961867 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:12.940136 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2699) took 1303 ms. Trace:
I20260709 12:19:12.962471 18300 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:12.939895 24605 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.914517 24539 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.967574 18448 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:19:13.003571 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50992.8us
I20260709 12:19:12.963909 18440 rpcz_store.cc:276] 0709 12:19:11.617740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.675653 (+ 57913us) service_pool.cc:224] Handling call
0709 12:19:11.701710 (+ 26057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.767473 (+ 65763us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:12.179367 (+411894us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.920642 (+741275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:12.920706 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:12.920763 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.920843 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.920997 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19648,"prepare.queue_time_us":289522,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":2882,"thread_start_us":981876,"threads_started":2}
I20260709 12:19:13.027562 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.949678 24707 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:12.967406 24645 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:19:13.039088 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:12.962332 24602 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.965646 24626 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:13.044631 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:12.967295 24644 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:13.053201 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:13.056461 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.059474 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:12.988143 24529 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.959489 24674 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.074532 19040 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:13.074898 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:13.082301 18375 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.040544 24736 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:13.081386 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 76562.9us
I20260709 12:19:13.079422 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:13.091112 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:13.084417 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:13.053757 24602 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:13.099371 19044 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:13.034101 24726 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:19:13.113651 18448 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:13.047381 24283 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:19:13.115594 19039 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:13.165510 24222 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.117866 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:13.040006 24539 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.104141 24602 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.168902 24539 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.120592 18266 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:13.170174 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49689.1us
I20260709 12:19:13.171128 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.172574 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:13.173336 24732 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:13.173445 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.174094 24732 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.174567 24732 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:13.022624 24605 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.134851 19043 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:13.176919 24605 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.178486 24605 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:13.179831 24539 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:13.179694 24629 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:13.138370 24737 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:13.181795 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:13.140837 18443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.025s	sys 0.046s
W20260709 12:19:13.200644 18443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.026s	sys 0.046s
I20260709 12:19:13.145545 18367 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:13.143826 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 196067us
I20260709 12:19:13.080849 24626 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.153175 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 2367) took 1142 ms. Trace:
I20260709 12:19:13.157255 24525 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.158614 18375 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:13.161424 24645 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.163635 24685 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.179661 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:13.138022 24731 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.707s	user 0.000s	sys 0.004s
I20260709 12:19:13.217257 19007 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:13.140118 24407 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.224272 24407 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.052366 18300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:13.227521 18300 raft_consensus.cc:606] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:13.227789 18366 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.016s	sys 0.022s
I20260709 12:19:13.202355 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:13.228171 18366 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.016s	sys 0.022s
W20260709 12:19:13.147583 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.203797 18372 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:13.098961 24736 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.205299 18297 rpcz_store.cc:276] 0709 12:19:12.011289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.011519 (+   230us) service_pool.cc:224] Handling call
0709 12:19:13.152906 (+1141387us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54000721d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:13.153079 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973439360}
I20260709 12:19:13.207274 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2696) took 1590 ms. Trace:
I20260709 12:19:13.180760 24629 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.212690 19040 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:19:13.214344 18447 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
W20260709 12:19:13.213222 24588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.012s	sys 0.023s
W20260709 12:19:13.173280 24746 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:13.218456 24724 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.218976 24222 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.219493 24726 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.175043 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:13.241154 18446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.018s	sys 0.041s
I20260709 12:19:13.129791 24335 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.179126 24605 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:19:13.243156 18412 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.014s	sys 0.044s
I20260709 12:19:13.216387 24683 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:13.250545 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.236971 24588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.012s	sys 0.023s
I20260709 12:19:13.244156 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:13.226996 24407 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.274111 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:13.242319 24011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.024s
I20260709 12:19:13.240602 24525 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:13.285359 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.286563 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69380us
W20260709 12:19:13.244197 18446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.018s	sys 0.041s
I20260709 12:19:13.237903 24629 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:13.248976 18412 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.014s	sys 0.044s
I20260709 12:19:13.296793 18297 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:13.294420 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 552034us
W20260709 12:19:13.282707 18296 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.012s	sys 0.037s
I20260709 12:19:13.283603 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:13.248873 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 73875.3us
W20260709 12:19:13.275866 24011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.013s	sys 0.024s
I20260709 12:19:13.241950 24283 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.216188 24645 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.304167 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:13.298933 18296 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.013s	sys 0.038s
I20260709 12:19:13.297245 18297 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:13.301537 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.283766 24728 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.255395 24635 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:13.292713 24632 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.295045 24529 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:13.327561 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2672) took 1735 ms. Trace:
I20260709 12:19:13.284808 24407 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.324966 24644 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:13.283062 24727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.031s	sys 0.020s
I20260709 12:19:13.276852 24631 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:13.249903 24602 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.255753 24202 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:19:13.295745 24737 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.326676 18297 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.360320 18297 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.245298 24726 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.343709 19039 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:13.304091 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:13.332297 24635 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:13.361074 24727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.032s	sys 0.020s
W20260709 12:19:13.351812 18291 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.000s	sys 0.044s
I20260709 12:19:13.336247 18371 rpcz_store.cc:276] 0709 12:19:11.592339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.592806 (+   467us) service_pool.cc:224] Handling call
0709 12:19:11.621973 (+ 29167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.635770 (+ 13797us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:11.635803 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:11.650564 (+ 14761us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.774401 (+123837us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b18004b9640. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:19:12.015428 (+241027us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:12.015464 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.015534 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.137444 (+1121910us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:13.327350 (+189906us) raft_consensus.cc:1685] finished
0709 12:19:13.327374 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.327502 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13341,"raft.queue_time_us":13931,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":13085,"spinlock_wait_cycles":50326656,"thread_start_us":240840,"threads_started":1,"wal-append.queue_time_us":737932}
W20260709 12:19:13.383086 18291 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.000s	sys 0.045s
W20260709 12:19:13.384112 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 564573us
I20260709 12:19:13.327402 24283 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.427899 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.435158 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2695) took 1818 ms. Trace:
I20260709 12:19:13.436362 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:13.437898 18293 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.016s	sys 0.019s
W20260709 12:19:13.440795 18293 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.016s	sys 0.019s
W20260709 12:19:13.446175 19038 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.000s
I20260709 12:19:13.446607 19038 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
W20260709 12:19:13.329314 24593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.010s	sys 0.014s
W20260709 12:19:13.376554 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 72561.7us
W20260709 12:19:13.450470 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.377381 18368 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.025s	sys 0.021s
W20260709 12:19:13.451700 18368 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.025s	sys 0.021s
I20260709 12:19:13.393651 24635 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.372073 24728 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.372649 24588 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:13.465361 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.417249 24628 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:13.466472 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:13.435591 18442 rpcz_store.cc:276] 0709 12:19:11.616437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.618156 (+  1719us) service_pool.cc:224] Handling call
0709 12:19:11.618519 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:11.636025 (+ 17506us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:11.965498 (+329473us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.760602 (+795104us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:12.761172 (+   570us) log.cc:844] Serialized 52 byte log entry
0709 12:19:12.761301 (+   129us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:12.761331 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.761400 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.434939 (+673539us) raft_consensus.cc:1685] finished
0709 12:19:13.434961 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.435096 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34439,"prepare.queue_time_us":392028,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":57027,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":16310,"thread_start_us":966132,"threads_started":2,"wal-append.queue_time_us":1043567}
I20260709 12:19:13.480542 18447 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:13.483018 24728 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.365706 24407 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.362927 24644 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.370095 24631 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.434545 24674 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.430541 24626 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.369035 24737 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.491370 24621 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.003s
I20260709 12:19:13.463202 24744 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.494051 24745 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:13.485265 24628 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:13.467973 24593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.010s	sys 0.016s
I20260709 12:19:13.492211 24335 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:13.513073 24755 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:19:13.514323 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2380) took 1017 ms. Trace:
I20260709 12:19:13.514876 18292 rpcz_store.cc:276] 0709 12:19:12.496597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.497244 (+   647us) service_pool.cc:224] Handling call
0709 12:19:12.497604 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.497859 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.497883 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.498346 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.982208 (+483862us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:12.982247 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.982304 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.514123 (+531819us) raft_consensus.cc:1685] finished
0709 12:19:13.514144 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.514267 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":482994,"threads_started":1,"wal-append.queue_time_us":825529}
W20260709 12:19:13.516814 24753 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:19:13.517372 24754 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:13.519639 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.534176 19038 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:13.518146 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2373) took 1242 ms. Trace:
I20260709 12:19:13.536135 18298 rpcz_store.cc:276] 0709 12:19:12.275631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.275793 (+   162us) service_pool.cc:224] Handling call
0709 12:19:12.364615 (+ 88822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.405156 (+ 40541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.405186 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.415255 (+ 10069us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.938471 (+523216us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:12.960446 (+ 21975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.960531 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.517919 (+557388us) raft_consensus.cc:1685] finished
0709 12:19:13.517949 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.518081 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":480612,"threads_started":1,"wal-append.queue_time_us":804431}
I20260709 12:19:13.540091 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.549425 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.510627 24628 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.540519 24754 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.535900 24745 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:13.561228 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:13.558727 18294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.008s	sys 0.004s
I20260709 12:19:13.517288 24610 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:13.602301 18294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.008s	sys 0.004s
I20260709 12:19:13.544150 24707 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.608871 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.507179 24621 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.507583 24626 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:19:13.509233 24588 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.616257 24756 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.551350 24749 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:13.509778 24744 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.629984 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 377532us
I20260709 12:19:13.633790 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.635300 19041 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
W20260709 12:19:13.628343 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 355247us
W20260709 12:19:13.641574 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 262681us
I20260709 12:19:13.642316 19039 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:13.626631 24728 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:13.648346 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.614265 24645 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.642767 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2432) took 1762 ms. Trace:
I20260709 12:19:13.646561 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:13.629338 24602 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:13.650843 18368 rpcz_store.cc:276] 0709 12:19:11.880536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.880836 (+   300us) service_pool.cc:224] Handling call
0709 12:19:11.886295 (+  5459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.897667 (+ 11372us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:12.124137 (+226470us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.452209 (+1328072us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.641105 (+188896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.641176 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:13.641237 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.641322 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.642711 (+  1389us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":568391,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":3118,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4805,"spinlock_wait_cycles":731904,"thread_start_us":1366972,"threads_started":3,"wal-append.queue_time_us":910508}
I20260709 12:19:13.658663 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.624873 24707 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:13.638443 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 336983us
I20260709 12:19:13.659832 19039 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
W20260709 12:19:13.662212 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 355949us
I20260709 12:19:13.634265 24756 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.666376 18412 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.677517 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.688719 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.690460 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1185) took 1017 ms. Trace:
I20260709 12:19:13.656100 24283 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.696029 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:13.699680 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 138519us
I20260709 12:19:13.690912 18291 rpcz_store.cc:276] 0709 12:19:12.672658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.672990 (+   332us) service_pool.cc:224] Handling call
0709 12:19:12.673283 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.673510 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:13.383557 (+710047us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.690001 (+306444us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.690092 (+    91us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:13.690147 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.690228 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.690392 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35517,"prepare.queue_time_us":833435,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":28275,"thread_start_us":905668,"threads_started":2}
I20260709 12:19:13.707924 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.708392 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.704986 18264 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:13.714133 18295 raft_consensus.cc:2368] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
W20260709 12:19:13.712113 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45855.8us
I20260709 12:19:13.657428 24728 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 12:19:13.720134 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.716101 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.717697 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.715577 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:13.693413 24726 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:13.726122 24759 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:19:13.675807 24616 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.691726 24525 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.727591 24784 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:13.727833 24774 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:19:13.724269 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:13.724470 24767 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:13.731056 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:13.710618 24610 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.714381 24602 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.709188 24786 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.735576 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1195) took 1001 ms. Trace:
I20260709 12:19:13.725545 24795 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:19:13.722386 24788 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:13.712649 24725 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:19:13.719761 24674 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.749229 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:13.754289 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.722636 24283 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:13.746464 18297 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:19:13.775853 18297 pending_rounds.cc:85] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 12
I20260709 12:19:13.776170 18297 pending_rounds.cc:107] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:19:13.776535 18297 raft_consensus.cc:2889] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:13.757212 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:13.731103 24737 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.762559 24663 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:13.763283 24777 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:13.757061 24645 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:19:13.679574 24745 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.766263 24732 consensus_meta.cc:287] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.002s
I20260709 12:19:13.768329 24744 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:13.768508 24707 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.784698 24756 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:13.770068 24782 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:13.727054 24616 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.770826 24781 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:13.771644 24758 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:19:13.778872 24796 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.778626 24283 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.799455 24335 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:13.749038 24610 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.773025 24685 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 12:19:13.783210 24787 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:13.777793 18447 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:13.749260 24644 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.748747 18296 rpcz_store.cc:276] 0709 12:19:12.734417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.747780 (+ 13363us) service_pool.cc:224] Handling call
0709 12:19:12.748322 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.751629 (+  3307us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:13.355120 (+603491us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.723365 (+368245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.723471 (+   106us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:13.735154 (+ 11683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.735291 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.735475 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3016,"prepare.queue_time_us":800232,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":9319,"spinlock_wait_cycles":5053184,"thread_start_us":843583,"threads_started":2}
I20260709 12:19:13.800423 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.820513 18294 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:13.804785 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.769033 24588 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:13.806823 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90292.8us
W20260709 12:19:13.809536 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 74315.1us
I20260709 12:19:13.941234 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:13.809497 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:13.757853 24788 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:13.789160 24784 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:13.813369 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 119282us
I20260709 12:19:13.783675 24202 consensus_queue.cc:1059] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:19:13.821270 18297 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:13.829164 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:13.812570 24781 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.955915 24781 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:13.234542 18443 rpcz_store.cc:276] 0709 12:19:11.616765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.674289 (+ 57524us) service_pool.cc:224] Handling call
0709 12:19:11.681737 (+  7448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.682081 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:12.183732 (+501651us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.206691 (+1022959us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.206881 (+   190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.206931 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:13.206987 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.207069 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.207193 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4331,"prepare.queue_time_us":347166,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":8262,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":25051,"spinlock_wait_cycles":2404224,"thread_start_us":1153092,"threads_started":2}
I20260709 12:19:13.956633 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.957738 18447 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:13.958050 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:13.841248 24629 consensus_meta.cc:287] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.517s	user 0.004s	sys 0.006s
I20260709 12:19:13.830677 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.849056 24616 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.959154 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.851929 24283 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.852523 24782 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:13.815600 24335 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:13.960254 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.960981 18292 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:13.960526 24335 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.961746 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:13.961867 24335 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.856436 24539 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.858916 24610 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:13.862401 19037 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:13.864663 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2694) took 2248 ms. Trace:
I20260709 12:19:13.800607 24777 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:13.967648 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 137064us
I20260709 12:19:13.968318 18366 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:13.968425 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.821113 24754 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.805663 24728 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:19:13.970074 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.802696 24732 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.971805 24539 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:13.972352 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 143245us
I20260709 12:19:13.972491 24732 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.878068 24803 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.973292 24803 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.973680 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:13.973888 24539 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.975641 24674 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:13.976074 24674 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.976274 24593 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.976392 18292 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:13.977006 24602 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.977381 24593 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.977576 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.894212 24628 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.977481 24602 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.978217 24593 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:13.978708 19041 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:13.979754 24736 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:13.980352 24628 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.900190 24635 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.908098 24514 raft_consensus.cc:993] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:13.980818 24782 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:13.981215 24514 raft_consensus.cc:1081] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:13.981374 24407 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.981719 24807 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:13.981227 24782 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:13.982447 24621 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.982940 24784 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.983022 24782 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:13.984042 24803 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.984432 24782 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:13.890564 24724 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.002s
I20260709 12:19:13.918929 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2790) took 1101 ms. Trace:
W20260709 12:19:13.985038 24768 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.334s	user 0.003s	sys 0.002s
W20260709 12:19:13.911048 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.926009 24737 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:13.932868 24609 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.910895 24786 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.953238 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.953810 18295 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:13.954123 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 229918us
I20260709 12:19:13.954180 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.952667 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:13.952952 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:13.952647 18263 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:13.954193 24788 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.958731 24629 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.827109 24707 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:19:13.962968 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.023351 18449 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.963362 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:13.964766 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:13.965161 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:13.966804 24725 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.967355 24632 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:13.969412 24777 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:13.985212 18293 rpcz_store.cc:276] 0709 12:19:12.817128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.826010 (+  8882us) service_pool.cc:224] Handling call
0709 12:19:12.858628 (+ 32618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.906233 (+ 47605us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:13.453552 (+547319us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.692478 (+238926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:13.692529 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:13.692590 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.918707 (+226117us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b54000aa0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:13.918753 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.918876 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1904,"prepare.queue_time_us":679084,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":27527,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11283,"spinlock_wait_cycles":108994432,"thread_start_us":729735,"threads_started":2,"wal-append.queue_time_us":318826}
I20260709 12:19:13.976476 24674 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:13.810642 24744 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.983057 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:13.987984 18369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2444) took 1349 ms. Trace:
I20260709 12:19:14.076558 18444 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.077189 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.079681 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.082973 19006 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:14.072919 24752 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.739s	user 0.000s	sys 0.006s
I20260709 12:19:14.076741 18369 rpcz_store.cc:276] 0709 12:19:12.638765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.662386 (+ 23621us) service_pool.cc:224] Handling call
0709 12:19:12.662706 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.826974 (+164268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.827019 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.829530 (+  2511us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.326725 (+497195us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:13.326768 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.438995 (+112227us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400115654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:13.439044 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.984369 (+545325us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:13.987711 (+  3342us) raft_consensus.cc:1685] finished
0709 12:19:13.987740 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.987911 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1738,"raft.queue_time_us":2239,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":375850624,"thread_start_us":496807,"threads_started":1,"wal-append.queue_time_us":782771}
I20260709 12:19:14.086340 18449 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:14.087589 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.088341 18449 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:13.991711 18441 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:13.992552 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.017657 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:13.986737 18292 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:14.018677 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.018713 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.019352 18291 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
W20260709 12:19:14.021571 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 68982.1us
I20260709 12:19:14.090225 18374 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.090746 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:14.091121 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 108113us
I20260709 12:19:14.092145 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:13.964529 24644 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:14.019590 18296 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:14.025007 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.101656 18412 leader_election.cc:400] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:14.101740 18372 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:14.102458 18412 leader_election.cc:403] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:13.964797 18446 rpcz_store.cc:276] 0709 12:19:11.616096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.635677 (+ 19581us) service_pool.cc:224] Handling call
0709 12:19:11.674402 (+ 38725us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:11.804557 (+130155us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:12.467764 (+663207us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.300850 (+833086us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.331472 (+ 30622us) log.cc:844] Serialized 52 byte log entry
0709 12:19:13.331628 (+   156us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:13.331663 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.331731 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.864366 (+532635us) raft_consensus.cc:1685] finished
0709 12:19:13.864404 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.864574 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23371,"prepare.queue_time_us":239928,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":37699,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":16184,"spinlock_wait_cycles":1665792,"thread_start_us":1242611,"threads_started":2,"wal-append.queue_time_us":957206}
I20260709 12:19:13.970806 18300 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:14.026947 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61861.1us
I20260709 12:19:14.020844 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:14.028231 24630 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:14.028579 24725 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:13.992648 24737 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:14.030385 24769 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.395s	user 0.004s	sys 0.004s
I20260709 12:19:13.990219 24796 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.020190 24609 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.038559 24781 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.039538 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.040952 24707 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:14.100687 24588 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.101269 24644 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:14.071408 19035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:14.132536 18374 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:14.134400 19035 raft_consensus.cc:606] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:14.071883 24776 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.694s	user 0.005s	sys 0.002s
I20260709 12:19:13.980182 24602 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.018657 18294 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:14.104035 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.107129 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.111414 24787 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:14.138435 18448 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.108343 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.108434 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.112112 18372 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:19:14.117477 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.143935 18292 raft_consensus.cc:2376] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16. Re-sending same reply.
I20260709 12:19:14.144531 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.111542 18440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:14.085026 24632 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.120113 24779 consensus_queue.cc:1059] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:14.122310 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:14.095618 24683 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.124997 18449 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:14.126878 24795 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:14.127210 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 106488us
I20260709 12:19:14.066226 24756 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.100878 24724 raft_consensus.cc:2764] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:14.067382 24629 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.109406 24630 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:14.145823 18440 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:14.163296 24630 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.094882 24744 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.165696 24744 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:14.147378 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.167619 18443 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:14.167743 18374 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:14.148231 18448 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:14.096254 24749 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:19:14.150591 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.171981 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1469) took 1144 ms. Trace:
I20260709 12:19:14.172688 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.173300 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.174042 18295 raft_consensus.cc:2368] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17.
I20260709 12:19:14.174212 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.152963 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:14.153717 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 45440.3us
I20260709 12:19:14.137190 24644 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:14.177362 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:14.161960 24756 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.162854 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:14.172398 18441 rpcz_store.cc:276] 0709 12:19:13.027126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.027264 (+   138us) service_pool.cc:224] Handling call
0709 12:19:14.171804 (+1144540us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54002044d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:14.171901 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215632640}
I20260709 12:19:14.148132 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:14.167832 18299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2791) took 1346 ms. Trace:
I20260709 12:19:14.182435 18299 rpcz_store.cc:276] 0709 12:19:12.820935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.826076 (+  5141us) service_pool.cc:224] Handling call
0709 12:19:12.856961 (+ 30885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.857285 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.857311 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:12.857760 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.342638 (+484878us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:13.342698 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.342770 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.030120 (+687350us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b240017fd68. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dce09d 00007f3c68a0010e 00007f3c68a1788c 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221
0709 12:19:14.167550 (+137430us) raft_consensus.cc:1685] finished
0709 12:19:14.167573 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.167775 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":29328768,"thread_start_us":484504,"threads_started":1,"wal-append.queue_time_us":690345}
I20260709 12:19:14.128026 24707 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:14.185164 24707 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.147977 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.175469 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.178099 18294 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:14.163417 18440 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.153586 24602 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.197989 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1466) took 1435 ms. Trace:
I20260709 12:19:14.198495 18445 rpcz_store.cc:276] 0709 12:19:12.762091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.797975 (+ 35884us) service_pool.cc:224] Handling call
0709 12:19:12.990374 (+192399us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de55fc0 00007f3c6de40929 00007f3c6de403d2 00007f3c673b7bd4
0709 12:19:12.990627 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:12.991026 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.991066 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.048259 (+ 57193us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.379248 (+330989us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:13.379656 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 12:19:13.649602 (+269946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.649700 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.071316 (+421616us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:14.197746 (+126430us) raft_consensus.cc:1685] finished
0709 12:19:14.197774 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.197924 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":209,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":104925,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":13067,"spinlock_wait_cycles":96514560,"thread_start_us":533884,"threads_started":2,"wal-append.queue_time_us":288704}
I20260709 12:19:14.201702 24632 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:14.202152 24632 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:14.202535 24632 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.205243 24781 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:14.207672 24635 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.158790 24588 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:14.211867 24737 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.213183 24737 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.149612 24539 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:14.221122 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8903d70080d24236be955433a92791c7"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:14.221746 18293 raft_consensus.cc:606] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:14.222123 18293 raft_consensus.cc:614] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:14.174814 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:14.232118 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55036us
W20260709 12:19:14.232344 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110139us
I20260709 12:19:14.233021 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:14.233354 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:14.234447 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.235056 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.235767 18444 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:14.234607 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:14.237418 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 89364.2us
I20260709 12:19:14.240540 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.240792 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.241395 18449 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.243662 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.244419 18448 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:14.128763 24796 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.246716 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.247406 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.248464 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.248973 18264 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:14.249395 18448 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:14.251187 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.251581 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.247009 19007 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:14.252171 18374 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.253259 18373 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:14.253887 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:14.259935 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.270764 18449 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:14.270912 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.272651 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.276355 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.284425 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:14.260798 18445 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:14.308136 18372 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.303726 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.309870 18367 raft_consensus.cc:2368] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
I20260709 12:19:14.299172 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:14.296139 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:14.310832 18374 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:14.311157 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:14.303614 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63842.5us
I20260709 12:19:14.312543 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.221760 24631 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.314064 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.249982 24707 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.314893 18446 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:14.250736 24663 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:14.317258 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69950.4us
I20260709 12:19:14.273792 24626 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:14.323108 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:14.254411 24635 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:14.261613 24749 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:14.327890 18338 leader_election.cc:400] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17.
I20260709 12:19:14.299063 24645 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:14.262054 24629 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:14.267917 24803 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:14.334342 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:14.276614 24202 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:14.337092 18338 leader_election.cc:403] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:14.321905 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:14.355743 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.356575 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:14.324903 24628 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:14.245641 24802 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 12:19:14.307457 24621 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:14.278062 24795 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:14.331925 24609 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:14.318538 24605 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.329003 24616 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:14.365190 18372 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:14.336680 24801 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:14.334978 24631 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:14.376549 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63563us
W20260709 12:19:14.379400 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 204008us
I20260709 12:19:14.333488 24626 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:14.383819 19041 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 21: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:19:14.318401 18183 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:14.374094 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46357.1us
I20260709 12:19:14.421787 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.422112 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.368680 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.342845 24663 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:14.394085 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.394361 24602 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:14.368314 24590 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:14.355182 24202 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:14.421197 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.429286 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.367954 24629 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:14.446849 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:14.343461 24635 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.439200 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:14.384307 24628 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:14.385059 24736 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17.
I20260709 12:19:14.384752 24724 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:14.442410 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.437723 24590 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.444247 24803 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:14.455336 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.447592 19038 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:14.456029 18370 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.380193 24621 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:14.449000 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 136542us
I20260709 12:19:14.450291 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:14.463671 19037 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.464586 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.436739 24645 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.451416 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:14.421382 24626 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.445766 24782 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:14.467703 18445 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.446061 24616 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.445906 24202 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:14.475564 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.435624 24749 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:14.433681 18183 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 118248us
I20260709 12:19:14.485373 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.455710 24786 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:14.463661 24590 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:14.453218 24725 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.483150 24628 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.492877 24663 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.494470 24629 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.495332 24616 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:14.496044 24816 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:14.496625 24808 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.497149 24808 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.497912 24616 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:14.475296 24724 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:14.498528 24616 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:14.508661 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:14.524060 18370 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:14.523931 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.526122 18444 raft_consensus.cc:2368] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17.
I20260709 12:19:14.456969 24788 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.530920 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:14.486006 24736 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.536103 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.537209 18375 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:14.498728 24805 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.488165 24786 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:14.479368 24630 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.548629 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.523686 24795 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.485770 24782 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.558703 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.559592 18375 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:14.528729 24822 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:14.541059 24804 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.544759 24796 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.563961 24621 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.566004 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:14.567776 24813 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:19:14.567785 19043 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.569501 24788 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:14.572587 24630 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:14.572970 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.546903 24786 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.554526 24795 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.585819 18440 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.587567 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:14.647715 19037 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:14.544962 24736 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:14.596459 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.655526 18441 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:14.599529 18445 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
W20260709 12:19:14.545540 24784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.000s	sys 0.019s
I20260709 12:19:14.528223 24749 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.561802 24808 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:14.671757 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:14.673125 18375 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.513906 24724 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.680699 18440 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:14.669274 24784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.000s	sys 0.019s
W20260709 12:19:14.687011 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 136733us
I20260709 12:19:14.603729 24635 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:14.636032 24616 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:14.567171 24632 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.688141 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:14.609652 24779 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:14.733069 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 45024.1us
W20260709 12:19:14.731688 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 339529us
I20260709 12:19:14.733294 18375 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
W20260709 12:19:14.701380 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 259035us
I20260709 12:19:14.735826 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:14.692617 24826 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:14.653803 24804 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.759970 24616 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:14.580025 24803 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:14.801889 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:14.809464 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73719.6us
I20260709 12:19:14.849614 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:14.852918 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.853603 18370 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
W20260709 12:19:14.802806 24815 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 12:19:14.883858 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 82049.5us
W20260709 12:19:14.868561 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 283052us
I20260709 12:19:14.784070 24779 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:14.936167 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.946273 18375 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:14.884670 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:14.970559 19009 leader_election.cc:400] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
I20260709 12:19:14.970950 19009 leader_election.cc:403] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:14.971922 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 87346.5us
I20260709 12:19:14.973239 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:14.981065 18373 consensus_meta.cc:287] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
I20260709 12:19:14.981647 18373 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:14.983528 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.885360 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:14.943769 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91952.2us
I20260709 12:19:14.989919 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:14.935724 24832 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:19:14.852176 24836 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:14.993884 18370 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:14.851572 24616 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:14.944890 24525 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.947156 24829 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:14.935739 24588 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.019735 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:14.988436 24801 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:15.031198 24745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.015s	sys 0.050s
I20260709 12:19:15.008817 24635 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:15.007673 24754 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:14.849445 24826 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.058643 18373 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:15.055248 24588 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.015820 24616 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.018095 24829 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.024699 24808 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:15.057235 24745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.200s	user 0.015s	sys 0.050s
I20260709 12:19:15.035156 24836 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:15.081632 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 232108us
I20260709 12:19:15.079373 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:15.060590 24707 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.000s
I20260709 12:19:15.093077 18373 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:15.093945 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:15.098023 18443 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:15.099643 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:15.080092 24829 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.023445 24525 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.107448 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:15.090312 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.218s (SENT) [suppressed 16 similar messages]
I20260709 12:19:15.108036 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:15.110879 24837 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:15.111258 24830 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.310s	user 0.000s	sys 0.002s
I20260709 12:19:15.112327 24804 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:15.170693 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 62723.9us
I20260709 12:19:15.129987 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:15.125746 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:15.104367 24844 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
W20260709 12:19:15.122462 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.173719 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:15.178896 19038 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:15.121660 24685 raft_consensus.cc:3055] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:15.137512 24539 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:15.143944 24839 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:15.120725 24785 consensus_queue.cc:1059] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
I20260709 12:19:15.163429 24782 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.122992 24632 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.127914 24525 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.201152 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:15.202695 24202 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.000s
W20260709 12:19:15.205440 24683 consensus_meta.cc:287] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.046s	user 0.007s	sys 0.004s
I20260709 12:19:15.206676 24781 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.188717 24736 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.188817 24685 raft_consensus.cc:740] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:15.224512 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:15.191691 24726 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:19:15.206744 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2698) took 1078 ms. Trace:
I20260709 12:19:15.247445 18368 rpcz_store.cc:276] 0709 12:19:14.128526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.128670 (+   144us) service_pool.cc:224] Handling call
0709 12:19:14.129061 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.129339 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:14.332664 (+203325us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.206234 (+873570us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.206423 (+   189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:15.206468 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:15.206509 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.206587 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.206690 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":873160,"prepare.queue_time_us":1948731,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":1986,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"thread_start_us":194547,"threads_started":1}
I20260709 12:19:15.175809 24850 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:15.192255 24811 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:15.193724 24539 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.206740 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2694) took 1110 ms. Trace:
W20260709 12:19:15.233533 24590 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.722s	user 0.008s	sys 0.003s
I20260709 12:19:15.231406 24683 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.169699 24837 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:15.230959 24645 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.003s
I20260709 12:19:15.227140 19039 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:15.231976 24782 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.233850 18369 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:15.266861 24514 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.247316 24525 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:15.242131 24736 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:15.255470 18366 rpcz_store.cc:276] 0709 12:19:14.096864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.097185 (+   321us) service_pool.cc:224] Handling call
0709 12:19:14.097569 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.097829 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:14.113014 (+ 15185us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.206133 (+1093119us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.206379 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:15.206442 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:15.206508 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.206587 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.206688 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1997724,"prepare.queue_time_us":961266,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":74,"raft.queue_time_us":1881,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":137923,"threads_started":1,"wal-append.queue_time_us":255245}
I20260709 12:19:15.253698 24811 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.253427 24850 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:15.277055 24827 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.472s	user 0.003s	sys 0.000s
I20260709 12:19:15.279302 24779 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.274302 24785 raft_consensus.cc:3055] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.297577 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:15.194326 24836 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.289106 24645 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.230504 24202 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.291590 24736 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.302629 18369 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
W20260709 12:19:15.299280 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 304902us
I20260709 12:19:15.329885 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.342478 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:15.330595 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.344108 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:15.294162 24811 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:15.342612 18447 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:15.284749 24590 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.355978 18367 raft_consensus.cc:1240] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.355829 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.287811 24782 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:15.320151 24674 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.224s	user 0.011s	sys 0.004s
I20260709 12:19:15.357784 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:15.341928 24736 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:15.298195 24635 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:15.293749 24685 consensus_queue.cc:260] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.437206 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.438071 18448 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:15.294041 24514 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.370584 24283 raft_consensus.cc:3055] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.378194 24756 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:15.381472 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:15.478847 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97455.9us
I20260709 12:19:15.382504 24745 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:15.295871 24525 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.304370 24785 raft_consensus.cc:740] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:15.444489 18447 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:15.352753 24683 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.320232 24588 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.448575 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:15.432332 19042 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:15.361937 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:15.458051 18449 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:19:15.463071 18448 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:19:15.506592 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 144732us
I20260709 12:19:15.293848 24850 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:15.464753 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 448517us
I20260709 12:19:15.507705 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:15.456792 24674 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.513005 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1121) took 1227 ms. Trace:
I20260709 12:19:15.435719 24826 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.514521 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:15.517328 19036 rpcz_store.cc:276] 0709 12:19:14.285684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.295385 (+  9701us) service_pool.cc:224] Handling call
0709 12:19:14.329443 (+ 34058us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.335463 (+  6020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.335484 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.341059 (+  5575us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.514263 (+173204us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:14.514323 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.514427 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.276442 (+762015us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:15.503007 (+226565us) raft_consensus.cc:1685] finished
0709 12:19:15.503027 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.512945 (+  9918us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5613,"raft.queue_time_us":5781,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":14627,"spinlock_wait_cycles":7273216,"thread_start_us":126679,"threads_started":1,"wal-append.queue_time_us":303334}
I20260709 12:19:15.461963 24283 raft_consensus.cc:740] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:15.520267 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:15.527714 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.499927 24726 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.532135 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.541929 19038 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:15.514181 24685 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:15.553290 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:15.553951 18296 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.518435 24756 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.555032 19041 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:19:15.520610 24588 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.393442 24796 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.520193 24822 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:15.557076 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.573397 18293 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:15.557816 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.548858 19037 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:15.520058 24803 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.571944 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.575076 18297 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:15.530778 24826 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.547686 24726 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.524031 24732 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.523782 24784 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:15.558936 24683 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.570860 24756 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.556521 24621 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.592192 18292 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:15.606145 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 515809us
I20260709 12:19:15.585850 24726 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.589722 24852 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.583127 24854 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:15.594620 24822 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:15.608495 24756 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.577802 24588 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.619771 24808 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.593384 24796 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.585037 24826 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.580829 24803 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.663841 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.587368 24283 consensus_queue.cc:260] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.580626 24635 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.609759 24784 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.609988 24683 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.602838 24732 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.669180 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.620338 24621 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.678026 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:15.680152 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.688952 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.690421 18297 raft_consensus.cc:2393] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:15.691677 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:15.648473 24829 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.695567 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.667783 24590 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:15.702483 24857 log.cc:927] Time spent T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:15.691527 24283 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.708922 24621 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.711125 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.711189 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.713570 18300 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:15.714660 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:15.717082 24683 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.670152 24782 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.699784 19037 raft_consensus.cc:606] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:15.723428 19037 raft_consensus.cc:614] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:15.643606 24852 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.668401 24837 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.701089 18266 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:15.720794 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.721272 18300 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:15.677418 24859 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:15.682158 24202 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.588811 24785 consensus_queue.cc:260] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.658413 24808 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.730434 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.726990 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.734437 18295 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:15.739660 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.741287 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.742185 18297 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:15.698128 24853 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:15.645964 24854 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.700430 24754 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:15.702523 24514 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:15.732204 24644 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 1.555s	user 0.009s	sys 0.009s
I20260709 12:19:15.770752 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2734) took 1509 ms. Trace:
I20260709 12:19:15.771301 18441 rpcz_store.cc:276] 0709 12:19:14.261581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.296903 (+ 35322us) service_pool.cc:224] Handling call
0709 12:19:14.297333 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.666804 (+369471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.770464 (+1103660us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54000d9654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:15.770682 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814093184}
W20260709 12:19:15.775616 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 204240us
I20260709 12:19:15.747972 24674 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.787472 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:15.724280 24822 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.739559 24859 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.782717 24861 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.806658 24861 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:15.806972 24786 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.128s	user 0.006s	sys 0.006s
W20260709 12:19:15.807060 24867 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:19:15.807706 24865 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:19:15.807524 24786 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:15.808246 24796 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.809520 18442 raft_consensus.cc:2393] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:15.763757 24853 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.810681 18442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2935) took 1417 ms. Trace:
I20260709 12:19:15.811942 24853 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:15.786799 24785 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.813336 18445 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.811014 18442 rpcz_store.cc:276] 0709 12:19:14.393630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.530602 (+136972us) service_pool.cc:224] Handling call
0709 12:19:15.810493 (+1279891us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400355cd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:15.810629 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2936078208}
I20260709 12:19:15.773212 24644 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.765117 24754 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.807119 24861 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:15.815685 24860 log.cc:927] Time spent T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:19:15.832422 18299 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:15.832332 18296 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:15.833829 18296 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [13.9-16.10]
W20260709 12:19:15.816162 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90239.1us
I20260709 12:19:15.812543 24853 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:15.842836 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55599.6us
W20260709 12:19:15.896907 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48304.5us
I20260709 12:19:15.898519 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:15.846758 24749 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:15.854763 19044 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:15.845994 24626 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.362s	user 0.013s	sys 0.000s
I20260709 12:19:15.844447 24786 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:15.846896 24836 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:15.844663 24826 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:15.847640 24796 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:15.942045 18294 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:15.925144 24869 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:19:15.909310 24837 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:15.849215 24588 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:15.921634 24810 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:15.956609 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 99195us
I20260709 12:19:15.898504 24853 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.922166 24871 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:15.925532 24784 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:15.944651 24880 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:19:15.984190 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.940394 24626 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:19:15.989327 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44260.4us
W20260709 12:19:16.022389 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 108848us
W20260709 12:19:15.956440 24616 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.866s	user 0.011s	sys 0.001s
I20260709 12:19:15.957938 24852 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.025576 24870 log.cc:927] Time spent T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 12:19:16.025826 24882 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:19:16.031284 24856 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:16.031823 24859 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.032377 24855 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:16.024077 24808 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:16.036151 24873 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:16.036476 18449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 2932) took 1643 ms. Trace:
I20260709 12:19:16.036481 24889 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:19:15.941298 24829 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.071559 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:15.945641 24796 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:15.984660 24635 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.087531 18449 rpcz_store.cc:276] 0709 12:19:14.392811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.392967 (+   156us) service_pool.cc:224] Handling call
0709 12:19:14.393279 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.506448 (+1113169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.036208 (+529760us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001a6154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.036407 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448016256}
I20260709 12:19:16.066512 18374 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:15.945019 24836 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.041867 18300 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:16.046983 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.066785 18447 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:16.268959 18293 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 12:19:16.107609 24883 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:19:16.118129 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.103595 24854 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.082872 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.087592 24856 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:16.116955 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:15.958739 24779 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.149796 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:16.151059 24804 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.151479 24202 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.003418 24810 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.074019 24786 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:16.154703 24796 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.004699 24749 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.123s
I20260709 12:19:16.073618 24843 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:15.998592 24871 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.156894 24635 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.042196 24616 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.024232 24732 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:16.093057 24866 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:19:16.037271 24539 consensus_meta.cc:287] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.749s	user 0.009s	sys 0.001s
I20260709 12:19:15.972870 24837 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.128069 24685 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.352842 24539 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.353005 24685 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.354200 24837 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.355005 24685 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.042616 24853 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.231662 24805 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.355758 24805 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.356673 24616 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.357760 24805 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.358686 24635 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:16.145807 24795 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.467s	user 0.014s	sys 0.000s
I20260709 12:19:16.081418 24884 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:16.262362 24756 consensus_meta.cc:287] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 12:19:16.145344 24644 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:16.286237 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.287549 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:16.331271 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 181504us
I20260709 12:19:16.333003 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.333550 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:16.289508 24873 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:16.337302 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.117143 24855 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.336730 24895 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.343019 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.343415 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:16.328516 24754 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.340898 24885 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 12:19:16.335804 24683 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:16.285377 24861 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.370105 18442 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:16.355216 24539 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.375480 18440 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 12: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:16.335352 24779 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.337688 24804 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.364010 24884 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.341362 24786 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.384677 18339 leader_election.cc:400] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17.
I20260709 12:19:16.339181 24202 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.393198 18449 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:16.400213 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.370309 24626 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.407286 24724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.010s	sys 0.008s
I20260709 12:19:16.374068 24895 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:16.408979 24891 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:19:16.402930 24884 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:16.415373 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.431375 18300 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:16.432572 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.433821 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.434420 18300 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:16.434775 18300 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:16.416625 18296 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:16.384111 24616 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.419158 18442 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:16.430553 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.439924 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.447953 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.454506 18300 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.461846 18300 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.463280 24871 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.464872 24779 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.465531 24872 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:16.467080 24804 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.363837 24795 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:16.486505 24888 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 12:19:16.359534 24856 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.366580 24756 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:16.478811 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 112605us
W20260709 12:19:16.481642 24785 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.001s
W20260709 12:19:16.460698 24724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.011s	sys 0.009s
W20260709 12:19:16.479636 24736 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 1.014s	user 0.008s	sys 0.004s
W20260709 12:19:16.485168 24850 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.956s	user 0.013s	sys 0.000s
I20260709 12:19:16.505100 24850 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.507257 24850 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.507442 18292 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:16.507871 18444 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
W20260709 12:19:16.402731 24726 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.759s	user 0.007s	sys 0.003s
I20260709 12:19:16.508563 24726 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.509280 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2690) took 2517 ms. Trace:
W20260709 12:19:16.510531 18372 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55012 (request call id 2701) took 2364 ms (client timeout 1641 ms). Trace:
W20260709 12:19:16.510895 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.511018 18372 rpcz_store.cc:269] 0709 12:19:14.146277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.147775 (+  1498us) service_pool.cc:224] Handling call
0709 12:19:16.510330 (+2362555us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001f9fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.510436 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271694720}
I20260709 12:19:16.422925 24837 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:19:16.512370 24786 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:16.512897 24837 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:16.513255 24837 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:16.504287 24736 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.502158 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:16.509658 18371 rpcz_store.cc:276] 0709 12:19:13.991761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.991867 (+   106us) service_pool.cc:224] Handling call
0709 12:19:14.047372 (+ 55505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.047642 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:14.048022 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.202399 (+1154377us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180044ae80. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c65d91a40 00007f3c65d8c4e4 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:19:15.552116 (+349717us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.552376 (+   260us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:15.552437 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:15.552504 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.552580 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:16.509109 (+956529us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b54002f7e50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.509219 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151912,"prepare.queue_time_us":1067556,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":4537,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":27765,"spinlock_wait_cycles":30055424,"thread_start_us":431525,"threads_started":2,"wal-append.queue_time_us":173881}
I20260709 12:19:16.519917 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.510607 24726 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.520579 19040 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:16.523623 19040 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:16.524531 18373 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:16.520490 24683 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.527155 24683 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.528810 24736 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.529727 24683 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.529985 18366 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 22: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:16.531073 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.514032 24837 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.541496 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.523571 24822 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:16.520457 24685 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.524024 24836 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.494010 24872 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.561866 18446 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:16.562389 18446 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:16.563411 18446 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.564640 18446 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.566892 24855 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:16.567677 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.568919 24861 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.569005 24836 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.569721 24836 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.570499 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.570822 24756 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:16.571130 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:16.572777 24795 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.573601 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.573164 24836 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.576923 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:16.577481 18447 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.582007 24872 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.559382 24685 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.584621 18447 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:16.585367 24685 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.586840 24811 consensus_meta.cc:287] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.199s	user 0.011s	sys 0.001s
I20260709 12:19:16.587311 24811 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.588411 18448 raft_consensus.cc:2368] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:16.591820 19043 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1482) took 1607 ms. Trace:
I20260709 12:19:16.592181 19043 rpcz_store.cc:276] 0709 12:19:14.984706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.984829 (+   123us) service_pool.cc:224] Handling call
0709 12:19:15.014807 (+ 29978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.090712 (+ 75905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.090745 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.097816 (+  7071us) log.cc:844] Serialized 26 byte log entry
0709 12:19:15.358119 (+260303us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:15.358158 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.591432 (+1233274us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b54003a0554. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.591476 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:16.591524 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:16.591538 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:16.591758 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14368,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11858,"spinlock_wait_cycles":173725952,"thread_start_us":196822,"threads_started":1,"wal-append.queue_time_us":464333}
I20260709 12:19:16.595853 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1483) took 1264 ms. Trace:
I20260709 12:19:16.555157 24822 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:16.596423 24803 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.596477 24811 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.597500 24822 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.601893 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2738) took 1272 ms. Trace:
I20260709 12:19:16.602293 18445 rpcz_store.cc:276] 0709 12:19:15.329802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.330075 (+   273us) service_pool.cc:224] Handling call
0709 12:19:15.813165 (+483090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.815085 (+  1920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.601683 (+786598us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400355cd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.601830 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063404928}
I20260709 12:19:16.596259 18444 rpcz_store.cc:276] 0709 12:19:15.331675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.355574 (+ 23899us) service_pool.cc:224] Handling call
0709 12:19:16.595694 (+1240120us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000d8754. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.595784 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2646168320}
I20260709 12:19:16.597952 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 52) took 1081 ms. Trace:
I20260709 12:19:16.610767 19037 rpcz_store.cc:276] 0709 12:19:15.516797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.519035 (+  2238us) service_pool.cc:224] Handling call
0709 12:19:16.597748 (+1078713us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54006bd854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.597868 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287473536}
I20260709 12:19:16.612933 24202 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.560185 24837 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:16.616140 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2461) took 1668 ms. Trace:
I20260709 12:19:16.616533 18374 rpcz_store.cc:276] 0709 12:19:14.947849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.947993 (+   144us) service_pool.cc:224] Handling call
0709 12:19:14.948343 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:16.263430 (+1315087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.615877 (+352447us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001676d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.616073 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571069824}
I20260709 12:19:16.619719 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.620571 18445 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:16.624007 24905 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.624675 24905 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.544313 24900 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.627036 24900 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.629539 24900 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.633136 18443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2924) took 2400 ms (client timeout 2218 ms). Trace:
W20260709 12:19:16.633584 18443 rpcz_store.cc:269] 0709 12:19:14.233152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.233305 (+   153us) service_pool.cc:224] Handling call
0709 12:19:16.632915 (+2399610us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.633031 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1896160128}
I20260709 12:19:16.635284 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2813) took 1108 ms. Trace:
I20260709 12:19:16.635704 18295 rpcz_store.cc:276] 0709 12:19:15.527271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.527413 (+   142us) service_pool.cc:224] Handling call
0709 12:19:16.635069 (+1107656us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54002747d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.635206 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473635712}
I20260709 12:19:16.636865 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.638505 24525 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.639098 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.639454 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.639976 18440 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:16.640638 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:16.640947 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.641376 18443 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:16.642172 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.642783 18448 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:16.643960 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.644726 18444 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:16.646950 24202 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:16.647328 18373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 2467) took 1557 ms. Trace:
I20260709 12:19:16.647553 24202 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.647718 18373 rpcz_store.cc:276] 0709 12:19:15.089903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.095710 (+  5807us) service_pool.cc:224] Handling call
0709 12:19:16.647096 (+1551386us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54000c8ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.647260 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3275188608}
I20260709 12:19:16.650622 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.651301 24202 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.652083 24892 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.494s	user 0.003s	sys 0.000s
I20260709 12:19:16.652218 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.652812 19037 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:16.656546 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.657130 18299 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:16.657809 24836 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:16.658381 24836 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.658695 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.659569 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.660162 18295 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.660604 24836 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:16.665010 19042 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 1118) took 2380 ms (client timeout 1859 ms). Trace:
W20260709 12:19:16.665432 19042 rpcz_store.cc:269] 0709 12:19:14.284586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.286219 (+  1633us) service_pool.cc:224] Handling call
0709 12:19:16.664796 (+2378577us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540055e6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.664915 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2525357312}
I20260709 12:19:16.666966 24811 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:16.668519 24811 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:16.668946 24811 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:16.669152 24836 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.669744 19007 leader_election.cc:400] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:16.669683 24836 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.669921 24811 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:16.670122 19007 leader_election.cc:403] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:16.672145 24836 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.672639 24836 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.672729 24685 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:16.673108 24836 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.675628 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.676337 18373 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:16.677331 24844 raft_consensus.cc:993] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:16.677822 24844 raft_consensus.cc:1081] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:16.679476 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.679605 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.680177 19042 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:16.681048 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.683331 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.683626 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.684139 18298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2810) took 1158 ms. Trace:
I20260709 12:19:16.684378 18370 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:16.685240 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.685848 18375 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:16.684525 18298 rpcz_store.cc:276] 0709 12:19:15.525600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.525714 (+   114us) service_pool.cc:224] Handling call
0709 12:19:15.551733 (+ 26019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.696780 (+145047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:15.696808 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:15.697855 (+  1047us) log.cc:844] Serialized 26 byte log entry
0709 12:19:15.830681 (+132826us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:15.830755 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.683766 (+853011us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400039dd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.683842 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:16.683890 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:16.683904 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:16.684085 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334514688,"thread_start_us":85321,"threads_started":1,"wal-append.queue_time_us":219139}
I20260709 12:19:16.687481 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.688028 18375 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:16.675730 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1300) took 2234 ms. Trace:
I20260709 12:19:16.688504 24810 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.689452 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.689855 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.690035 18375 raft_consensus.cc:2376] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20. Re-sending same reply.
I20260709 12:19:16.690517 18372 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:16.690976 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.688542 18367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2708) took 1649 ms. Trace:
I20260709 12:19:16.691551 18367 rpcz_store.cc:276] 0709 12:19:15.041584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.157925 (+116341us) service_pool.cc:224] Handling call
0709 12:19:15.174475 (+ 16550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.357089 (+182614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.688234 (+1331145us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54000fa354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.688462 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416933632}
I20260709 12:19:16.692740 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.693396 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:16.694289 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.694797 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2812) took 1168 ms. Trace:
I20260709 12:19:16.695569 18367 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:16.697590 24895 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.688896 19041 rpcz_store.cc:276] 0709 12:19:14.441541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.441678 (+   137us) service_pool.cc:224] Handling call
0709 12:19:14.455327 (+ 13649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:15.571015 (+1115688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.675460 (+1104445us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540066e4d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.675643 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528495360}
I20260709 12:19:16.694855 18372 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.700984 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1464) took 1187 ms. Trace:
I20260709 12:19:16.701344 18366 rpcz_store.cc:276] 0709 12:19:15.513986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.514297 (+   311us) service_pool.cc:224] Handling call
0709 12:19:16.700812 (+1186515us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400414454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.700932 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317952256}
W20260709 12:19:16.702327 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.680047 24835 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 1.515s	user 0.005s	sys 0.012s
I20260709 12:19:16.710120 18446 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:19:16.707856 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.708454 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:16.711290 18299 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:16.711674 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:16.708906 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.712563 18296 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.695302 18291 rpcz_store.cc:276] 0709 12:19:15.526286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.526426 (+   140us) service_pool.cc:224] Handling call
0709 12:19:15.526770 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.549870 (+ 23100us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:19:15.695748 (+145878us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.040094 (+344346us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.040284 (+   190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:16.040326 (+    42us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:19:16.040370 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.694549 (+654179us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b540004cc54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.694603 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:16.694735 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23400,"prepare.queue_time_us":165472,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":53154,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":32026,"spinlock_wait_cycles":52545792,"thread_start_us":455765,"threads_started":4,"wal-append.queue_time_us":245524}
I20260709 12:19:16.712015 18371 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:16.680148 18373 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:16.714421 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.714576 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.714718 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.714421 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:16.715936 18375 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:16.716248 19042 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:16.720372 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:16.722107 18300 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:16.715329 24736 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.728173 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.731755 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:16.732820 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.733438 18375 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.735455 18372 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:16.738723 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.741456 18372 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:16.751543 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.752226 19042 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:16.752521 18296 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:16.754040 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:16.754557 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2407) took 1062 ms. Trace:
W20260709 12:19:16.731770 24707 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:19:16.755147 18294 rpcz_store.cc:276] 0709 12:19:15.692216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.917687 (+225471us) service_pool.cc:224] Handling call
0709 12:19:15.918050 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.943470 (+ 25420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.754328 (+810858us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b540040e054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.754504 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51749632}
I20260709 12:19:16.726207 24848 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:16.743645 24630 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.760675 24810 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:16.761469 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:16.761657 24781 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.762637 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.764230 24884 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.753536 24626 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.764745 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.810173 18375 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:16.765723 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:16.811269 19041 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:16.768980 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1306) took 1640 ms. Trace:
I20260709 12:19:16.769950 18295 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:16.775689 24616 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.778131 24796 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.733031 24811 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.795912 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.737664 24736 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.731081 24838 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.828671 24838 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.802068 19037 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:16.764786 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:16.832218 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67509.6us
I20260709 12:19:16.832485 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:16.833168 18292 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:16.834208 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:16.801748 18264 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:16.836452 24896 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.836723 18264 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:16.814514 18371 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:16.838459 24805 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:16.838969 24805 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:16.814050 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.812155 19044 rpcz_store.cc:276] 0709 12:19:15.128681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.130183 (+  1502us) service_pool.cc:224] Handling call
0709 12:19:15.130483 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.903829 (+773346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.768701 (+864872us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b5400640ad4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:16.768907 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639647104}
I20260709 12:19:16.766980 24685 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:16.825781 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.848623 19042 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:19:16.800118 19008 connection.cc:256] RPC call timeout handler was delayed by -1.8094200580001143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:19:16.836902 24896 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:16.819334 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.840652 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:16.831037 24838 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.845330 19041 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:16.809381 24905 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.801867 24781 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:16.851529 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:16.872061 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.801412 24707 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.853572 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.824285 24616 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.814321 24848 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.864928 18446 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:16.865484 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.865901 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:16.867223 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:16.813365 24626 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:16.871614 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.808494 24801 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:19:16.801316 24630 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.816804 24795 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:16.839329 24878 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:19:16.826745 24796 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.846408 24879 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 12:19:16.873369 24788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.664s	user 0.012s	sys 0.024s
W20260709 12:19:17.040771 24788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.831s	user 0.012s	sys 0.024s
I20260709 12:19:16.895695 24833 raft_consensus.cc:2764] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:19:17.042372 19039 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:17.042850 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.043365 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.043565 19036 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:17.044582 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:17.045315 19039 raft_consensus.cc:2393] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:17.045653 24796 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:17.046063 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 180240us
I20260709 12:19:16.876410 19037 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:17.069778 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.978680 24884 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:16.868197 24836 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.935858 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:16.971206 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.071530 18366 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:17.072683 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.007588 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.073760 18366 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:17.074026 18370 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:16.849677 24628 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:17.075083 24628 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.075107 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.075698 24628 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:17.076004 18370 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:16.903153 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.079068 18371 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:19:17.081182 18371 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:17.082738 18371 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:17.083884 24884 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:17.025642 18196 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:16.935592 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:17.084425 24884 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:16.866091 24898 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.086195 24898 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.057734 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:16.873267 19041 raft_consensus.cc:2393] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:17.088845 18369 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 13.
I20260709 12:19:17.078045 18370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:19:16.908120 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:16.915779 24805 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:16.875489 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.091971 24616 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:16.960012 24856 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.956961 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:16.875049 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.099793 19043 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:16.981019 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.102458 18444 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:16.866659 24896 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.061268 18373 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:17.049736 24795 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.876825 24781 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.923317 18293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2402) took 1254 ms. Trace:
I20260709 12:19:16.933409 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.127095 18375 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:17.126876 18293 rpcz_store.cc:276] 0709 12:19:15.668844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.752771 (+ 83927us) service_pool.cc:224] Handling call
0709 12:19:16.265693 (+512922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.289088 (+ 23395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.922997 (+633909us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54002760d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:16.923164 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161762176}
I20260709 12:19:17.062806 24626 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.015300 24644 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.872565 24685 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.976502 24736 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.063884 24635 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:17.127370 24795 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:17.065327 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.066169 24605 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.863099 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.047272 19044 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:19:16.896185 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.911480 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.019714 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.086683 24884 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.050076 24906 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:17.032763 24843 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:17.090107 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 328700us
I20260709 12:19:17.154676 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.068753 24833 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:17.074647 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.092254 18446 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:17.087496 24707 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.090338 19042 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:17.007120 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:16.898555 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.168102 18374 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:17.093001 18196 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 20, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 20 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:17.110987 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.117344 24898 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.169597 19036 raft_consensus.cc:2410] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 12:19:17.170414 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.133140 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.134567 18373 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:17.136893 18444 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:17.138693 19009 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.065234 24630 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.172266 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.172724 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:17.173321 18293 raft_consensus.cc:606] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.173400 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.173755 18293 raft_consensus.cc:614] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:17.158797 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 295822us
I20260709 12:19:17.176798 18412 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.159030 24843 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.160961 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.162330 24836 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.143044 24644 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.164938 24848 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.166484 24628 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:17.171610 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 106350us
I20260709 12:19:17.159365 24890 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.183523 18264 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.188341 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.188582 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.179066 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.151935 24605 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:17.175001 24616 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:17.172859 24878 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.167476 24736 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.185999 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.192457 19039 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:17.174688 24796 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:17.177892 24822 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.191663 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.182543 24906 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:17.182049 24900 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:17.190330 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.197935 19009 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.185310 24630 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:17.187531 24856 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.193611 24837 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:17.175727 24833 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.206521 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.204883 18196 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:17.211575 24663 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:17.212105 24663 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:17.212196 24707 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:17.193949 24904 raft_consensus.cc:2764] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:17.198076 24878 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:17.213929 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.199182 24906 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.197391 24202 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.208333 18412 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.192451 24674 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:19:17.245239 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.222906 24896 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.208015 24913 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:17.204083 18338 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.202226 24900 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.219705 24834 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.225816 24805 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:17.206072 24837 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.202291 24796 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.214011 24890 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:17.254890 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.213165 24630 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:17.259874 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55932.1us
I20260709 12:19:17.270220 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.270946 18371 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 12.
I20260709 12:19:17.198685 24616 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.261186 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.230345 24878 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.209848 24822 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:17.243345 24906 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:17.264292 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.200147 24736 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.242274 24202 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.241825 24514 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:19:17.262769 24805 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:17.264163 24913 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:17.277532 24795 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.252359 24707 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.252208 24635 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:17.278802 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.308490 18373 raft_consensus.cc:2410] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:19:17.282172 24898 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:17.287556 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.289024 24644 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.289268 18368 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:17.261250 24904 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.262614 24900 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:17.263031 24788 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:17.264354 24837 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:17.247767 24663 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.301878 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:17.303090 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48285.3us
I20260709 12:19:17.354629 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.297791 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:17.309489 18375 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:17.229154 24605 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.296748 24810 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:17.314497 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 116643us
W20260709 12:19:17.298291 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84452.9us
I20260709 12:19:17.367106 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:17.368146 18441 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:17.303974 24202 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.305694 24805 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:17.353896 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.356859 18448 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:17.307860 24635 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.312139 24904 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.368535 18441 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:17.377364 19009 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.343801 24876 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.305595 24884 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 12:19:17.366087 24888 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:19:17.388825 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.379381 18373 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:17.410626 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.411180 18373 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:17.375658 24915 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:19:17.412266 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.416127 18367 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.366619 24788 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:17.366104 24810 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.386739 24635 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:17.309180 24630 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.451375 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
W20260709 12:19:17.414247 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47228.2us
I20260709 12:19:17.455945 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.386531 24707 raft_consensus.cc:2764] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:19:17.391665 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:17.474558 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 82992.8us
I20260709 12:19:17.475809 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.409162 24605 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.449270 24925 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.454484 18441 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.478606 18441 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:17.478700 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 57894.3us
I20260709 12:19:17.478883 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:17.436385 24924 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:19:17.467880 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 166125us
W20260709 12:19:17.480123 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 91369.7us
I20260709 12:19:17.386860 24822 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:19:17.389993 24904 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.450362 24514 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:17.453788 24810 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.480885 18263 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.487278 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.490151 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:17.547909 18442 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:17.492945 18373 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:17.465581 24843 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:17.509169 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.510594 24628 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.483141 24811 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:17.511960 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.520185 24905 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.435935 24922 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.498941 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.571101 18367 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:17.549912 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:17.560819 18441 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:19:17.561527 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 74343.2us
I20260709 12:19:17.573633 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.379060 24644 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.145s
I20260709 12:19:17.562175 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.522607 24632 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.521929 24833 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.451097 24876 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:17.568893 24835 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:19:17.585019 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 128143us
I20260709 12:19:17.567414 24811 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.573668 24707 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:17.416676 24913 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.441224 24788 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:17.590699 18373 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:17.577278 24843 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:17.609797 24616 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.609890 24843 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.570891 24922 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:17.610386 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 129767us
I20260709 12:19:17.611471 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.613045 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.605934 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.582103 24796 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.596405 24930 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:19:17.609187 24707 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:17.610574 24922 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.580173 24896 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.599121 24628 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:17.625372 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.653108 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.605726 24856 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.683478 24932 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:19:17.686159 24878 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:17.689049 24626 consensus_meta.cc:287] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.000s
I20260709 12:19:17.610409 24843 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:17.655958 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.606930 24811 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:17.645083 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71525.8us
W20260709 12:19:17.676928 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 92008.9us
I20260709 12:19:17.641703 24628 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:17.693799 24900 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.619313 24736 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.700523 18196 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 11, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 11 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:17.701452 18193 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 297) took 46284 ms (client timeout 15000 ms). Trace:
I20260709 12:19:17.701176 24202 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.675354 24803 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:17.604386 24795 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.669713 24707 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.641129 24889 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:17.679471 18338 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:17.706034 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63806.2us
I20260709 12:19:17.783571 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:17.679625 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54355.8us
W20260709 12:19:17.784279 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.724012 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 71000.2us
W20260709 12:19:17.781111 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.712215 24884 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:19:17.786090 18196 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:17.676179 24632 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:17.636445 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 157622us
I20260709 12:19:17.781188 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:17.785537 18263 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:17.787931 18196 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 15, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 15 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:17.642886 24935 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:17.741569 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.789492 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.788803 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:17.790115 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 110760us
I20260709 12:19:17.709822 24905 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:17.837852 24905 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.838253 24905 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:17.840951 24781 consensus_meta.cc:287] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.703s	user 0.005s	sys 0.005s
I20260709 12:19:17.841368 24781 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.842367 18372 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:17.737360 24829 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.845185 18370 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:17.845594 18370 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:17.703398 24844 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:17.703651 24801 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.823539 18196 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:17.826747 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.827775 19036 raft_consensus.cc:1240] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:17.849035 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.828544 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:17.848897 18196 catalog_manager.cc:5697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:17.850373 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 61642.5us
I20260709 12:19:17.851064 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:17.852080 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:17.783646 24836 consensus_meta.cc:287] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.001s
I20260709 12:19:17.829396 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.782655 24628 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:17.836962 24685 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.001s
I20260709 12:19:17.706746 24810 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:17.847824 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66732.7us
I20260709 12:19:17.761070 24945 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.854465 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.855551 18196 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:17.856648 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:17.856798 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 67372.9us
I20260709 12:19:17.857388 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:17.834863 24805 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.854022 24926 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.859126 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:17.859221 24796 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.859735 24836 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.859925 24876 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.910393 19039 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:17.856177 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:17.894332 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:17.862538 24906 consensus_meta.cc:287] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.001s
I20260709 12:19:17.782579 24943 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.895686 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:17.866854 18370 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:17.877971 18339 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.583s (SENT)
I20260709 12:19:17.916438 18196 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 14, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 14 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:17.781075 24803 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:17.926797 18372 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1468) took 1020 ms. Trace:
I20260709 12:19:17.896061 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:17.911404 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55292.9us
I20260709 12:19:17.822294 24884 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:17.928581 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.913048 18449 raft_consensus.cc:1240] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:17.929518 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:17.930384 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:17.932991 19039 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:17.863574 24889 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:17.913048 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.947840 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.948544 18445 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:17.913883 18441 raft_consensus.cc:2393] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 21: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:17.853380 24605 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.912258 18448 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:17.820622 24736 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:17.922334 18292 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:17.886202 24933 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:17.894882 24202 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.957346 24644 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:17.981680 24810 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.981504 24644 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.854998 24844 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:17.924959 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:17.925073 18196 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:17.872464 24829 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:19:17.989776 18196 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 20, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 20 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:17.763558 24514 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.927404 18367 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.937433 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.002604 19039 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 21: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:17.927167 18372 rpcz_store.cc:276] 0709 12:19:16.906230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.906383 (+   153us) service_pool.cc:224] Handling call
0709 12:19:16.906784 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.842732 (+935948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.926641 (+ 83909us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54001685d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:17.926757 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151187200}
I20260709 12:19:17.864084 24890 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.885072 24663 consensus_queue.cc:1059] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:19:18.004081 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.004151 24663 raft_consensus.cc:3055] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:18.004750 24663 raft_consensus.cc:740] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:18.004882 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.005143 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:18.006091 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.005782 24663 consensus_queue.cc:260] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.006793 18372 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:17.781276 18193 rpcz_store.cc:269] 0709 12:18:31.417532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.417883 (+   351us) service_pool.cc:224] Handling call
0709 12:19:17.701194 (+46283311us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:10.854008 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:10.854867 (+   859us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:10.854950 (+    83us) write_op.cc:435] Acquired schema lock
0709 12:19:10.854988 (+    38us) tablet.cc:662] Decoding operations
0709 12:19:10.855827 (+   839us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:10.855951 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:10.856000 (+    49us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:19:10.856659 (+   659us) tablet.cc:701] Row locks acquired
0709 12:19:10.856718 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:19:10.856993 (+   275us) write_op.cc:270] Start()
0709 12:19:10.857179 (+   186us) write_op.cc:276] Timestamp: P: 1783599550856916 usec, L: 0
0709 12:19:10.857229 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:19:10.991844 (+134615us) log.cc:844] Serialized 1302 byte log entry
0709 12:19:11.360925 (+369081us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5400030820. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68df1dbe 00007f3c68dfc930 00007f3c68dfc8d8 00007f3c68dfc858 00007f3c68dfc813 00007f3c68dfba9c 00007f3c6ece72f6 00007f3c6ece1aa8 00007f3c68e387cb 00007f3c68e3b2c0
0709 12:19:11.720445 (+359520us) op_driver.cc:464] REPLICATION: finished
0709 12:19:12.349453 (+629008us) write_op.cc:301] APPLY: starting
0709 12:19:12.380297 (+ 30844us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:12.438109 (+ 57812us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:12.438146 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:12.763390 (+325244us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:12.763656 (+   266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:19:12.763719 (+    63us) write_op.cc:312] APPLY: finished
0709 12:19:12.764015 (+   296us) log.cc:844] Serialized 41 byte log entry
0709 12:19:13.437956 (+673941us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:13.463374 (+ 25418us) write_op.cc:454] Released schema lock
0709 12:19:13.490247 (+ 26873us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:13.584074 (+ 93827us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:13.600800 (+ 16726us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21580,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":477141,"apply.run_cpu_time_us":60456,"apply.run_wall_time_us":1274280,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16655,"num_ops":3,"prepare.queue_time_us":326557,"prepare.run_cpu_time_us":18570,"prepare.run_wall_time_us":283556,"raft.queue_time_us":305682,"raft.run_cpu_time_us":47253,"raft.run_wall_time_us":274335,"replication_time_us":863055,"spinlock_wait_cycles":9659520,"thread_start_us":1355284,"threads_started":5,"wal-append.queue_time_us":862383,"wal-append.run_cpu_time_us":15602,"wal-append.run_wall_time_us":2480563}]]}
I20260709 12:19:17.893949 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:17.858609 24801 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:17.946146 18415 leader_election.cc:400] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:18.009495 18415 leader_election.cc:403] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:19:17.947039 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.932155 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:17.885851 24878 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.713400 24626 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.888672 24781 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.953522 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.019728 18449 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16.
I20260709 12:19:17.955250 18299 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:17.895355 24685 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.900174 24900 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.919023 24795 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.859251 24941 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.884450 24945 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.952739 24736 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:17.951686 24605 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.959964 24884 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:18.032701 24941 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:18.034745 24644 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:17.965195 24805 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.965289 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:17.969610 24843 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.979507 18370 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.984517 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.984787 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:17.926568 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.045993 18445 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:17.887355 24944 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:19:17.950461 24932 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.946502 24803 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:17.997005 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:17.893714 24635 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:17.999986 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.000634 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.010860 18196 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:18.012902 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 93124us
I20260709 12:19:18.034437 24663 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:17.994577 24837 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.002s
I20260709 12:19:18.071830 18448 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:18.025804 18367 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:18.042742 24781 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.032646 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.034936 24605 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.035678 24884 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:17.960592 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.037563 24736 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.037896 24626 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:17.934458 24943 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.039728 18366 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:17.959906 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:18.020839 24707 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:18.057705 24932 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:18.042601 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 81195.6us
W20260709 12:19:18.108894 24949 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:19:18.043114 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 77914.1us
I20260709 12:19:18.108098 24605 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.112660 18266 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.015259 24796 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:18.050465 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.113893 24805 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:18.114018 18445 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:18.114455 24805 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:18.053229 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68299.5us
I20260709 12:19:18.115413 24805 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.089068 24837 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.118136 24837 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.029961 24795 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:18.118837 18369 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:18.119252 24795 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.110934 24905 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.058727 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:18.137048 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78402us
I20260709 12:19:18.137811 18371 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 12.
I20260709 12:19:18.137985 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:18.013299 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 81226.5us
I20260709 12:19:18.139760 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:18.140093 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.140618 18371 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:18.039764 24941 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.059773 18196 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 14, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 14 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:18.013098 24801 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.075989 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:17.925915 24906 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.051316 24925 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:18.020556 24922 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.054335 24674 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:18.160748 24674 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.162942 24674 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.115964 18293 raft_consensus.cc:2393] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.120653 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:18.120959 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.164594 24905 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:18.164780 18445 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:18.165041 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 44472.2us
I20260709 12:19:18.165731 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:18.121795 18373 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:18.167493 24685 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.168179 24685 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:18.096895 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.170429 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.173717 18445 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:18.148062 24937 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:18.175697 24937 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:18.176496 24937 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.177660 24628 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.178699 19041 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:18.179767 24945 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.142948 24605 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:18.052542 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 104595us
I20260709 12:19:18.174705 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.120277 19038 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:18.058050 18441 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.134320 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.134693 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.135093 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.129518 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.020092 24833 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:19:18.142052 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.186581 24632 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:18.143922 18446 raft_consensus.cc:2376] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20. Re-sending same reply.
W20260709 12:19:18.186864 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44932.8us
I20260709 12:19:18.144132 18291 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:18.188216 24940 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:18.188333 19009 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.188678 24940 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.189108 24940 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:18.143582 18196 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 14, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 14 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.151727 24890 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:18.150559 18338 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.192346 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.193070 18375 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.190425 18196 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 18, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 18 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:18.154299 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.189126 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.152314 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.163388 24796 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:18.051450 19040 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:18.183327 24837 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:18.192956 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.200903 19035 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.108480 24843 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.195982 18196 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 14, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 14 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:18.109076 24943 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:18.217310 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.218444 18300 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:18.219522 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.220160 18300 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:18.221791 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.203989 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.204255 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.204974 18295 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:18.186990 24632 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:18.139312 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:18.211135 18196 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 10 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:18.216532 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.217265 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.223438 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.224395 18375 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:18.159303 24906 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:18.224939 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85752.8us
I20260709 12:19:18.226691 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.094391 24781 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.232515 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.232889 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.233428 18294 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:18.227918 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.234597 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.230809 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.199674 24674 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:18.236343 19039 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.060647 24844 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.190478 24856 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.244789 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.245342 18445 raft_consensus.cc:2376] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17. Re-sending same reply.
I20260709 12:19:18.246459 18374 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.208606 24788 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:18.200438 24837 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:18.273345 24837 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.239216 18298 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.278631 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:18.246351 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.240415 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:19:18.230409 24904 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.000s
I20260709 12:19:18.246380 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1479) took 1185 ms. Trace:
I20260709 12:19:18.283272 18366 rpcz_store.cc:276] 0709 12:19:17.060474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.074349 (+ 13875us) service_pool.cc:224] Handling call
0709 12:19:18.246135 (+1171786us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400115b54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:18.246281 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031224576}
I20260709 12:19:18.281265 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.287671 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:18.227835 18196 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:18.289672 18372 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:18.251763 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.289919 18196 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 15, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:18.252715 24707 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:18.236634 24905 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:18.229338 24957 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:18.255638 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.311383 19036 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.257875 24922 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.222543 24926 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:19:18.260079 19040 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 1124) took 1409 ms. Trace:
I20260709 12:19:18.224360 24626 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:18.264602 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.264914 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.267541 24628 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.271512 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.271358 24856 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:18.272858 24788 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:18.280913 18293 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:18.282358 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.291244 18196 catalog_manager.cc:5697] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 17, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:18.293578 24945 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:18.294221 24913 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:18.290647 18443 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:18.345790 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.296838 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.300877 19039 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:18.302467 18300 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:18.353782 18366 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.299441 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.354611 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.238381 24736 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.306603 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.357803 18299 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:18.358637 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.312081 19035 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:18.307848 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.314296 19040 rpcz_store.cc:276] 0709 12:19:16.851120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.851253 (+   133us) service_pool.cc:224] Handling call
0709 12:19:18.259909 (+1408656us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400693054. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:18.259994 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674123648}
I20260709 12:19:18.336514 18294 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
W20260709 12:19:18.359717 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51981.9us
W20260709 12:19:18.325338 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.342206 18196 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 13 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:18.360203 18263 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:18.361241 18196 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.364252 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.365514 18375 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:18.303097 24932 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.303324 24905 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:18.315861 24822 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:18.368531 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.369154 18294 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.364295 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.373179 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.376113 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.372210 18445 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:18.362851 18196 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 18, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:18.298666 24943 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:18.353159 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.355151 18300 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.305619 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.378207 18196 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:18.379755 19036 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:18.385907 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.405596 18443 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.386555 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46418 (request call id 2482) took 1065 ms. Trace:
I20260709 12:19:18.311115 24796 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:19:18.355636 24674 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.297752 24933 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:19:18.406291 18374 rpcz_store.cc:276] 0709 12:19:17.320818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.338080 (+ 17262us) service_pool.cc:224] Handling call
0709 12:19:18.246267 (+908187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:18.246885 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.386368 (+139483us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400115b50. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:18.386503 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087201152}
I20260709 12:19:18.298352 24904 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.433260 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:18.391623 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.410815 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.433993 18440 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.434427 18448 raft_consensus.cc:2368] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
I20260709 12:19:18.431694 18300 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:18.315897 24626 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:18.435865 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.436913 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.315687 24906 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:19:18.417224 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.418478 18299 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 18. Ops: []
I20260709 12:19:18.352953 24913 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:18.368018 24810 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:18.423148 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190829us
I20260709 12:19:18.422083 24674 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.374723 24905 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.429471 18449 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:18.440860 24844 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.400871 24932 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:18.317291 24957 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.442240 18294 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:18.443496 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.372674 24829 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:18.450641 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.470212 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:18.478070 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.479267 18446 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:18.480057 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.482115 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:18.483024 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 186610us
I20260709 12:19:18.485018 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.485999 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.487401 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.487993 18448 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:18.488816 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.489326 18448 raft_consensus.cc:3060] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.486084 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.490132 18440 raft_consensus.cc:2393] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.491760 18338 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.492588 18441 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.448987 24913 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:18.497385 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.498595 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.502902 18448 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:18.503749 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.504171 18448 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.493781 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:18.508561 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.518071 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.526669 19039 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.528538 19007 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.533978 18294 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:18.534358 18294 pending_rounds.cc:85] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 9
I20260709 12:19:18.534561 18294 pending_rounds.cc:107] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:18.534783 18294 raft_consensus.cc:2889] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.477263 24943 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.513938 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:18.548646 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.549360 19036 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:18.549918 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.550629 19039 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:18.551558 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.552238 18299 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
W20260709 12:19:18.552627 24952 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 12:19:18.553156 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.552275 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.554425 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1505) took 1243 ms. Trace:
I20260709 12:19:18.554790 18444 rpcz_store.cc:276] 0709 12:19:17.311167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.311303 (+   136us) service_pool.cc:224] Handling call
0709 12:19:17.352333 (+ 41030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:17.352663 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:17.598550 (+245887us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b18002da960. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c6c8743c7 00007f3c68e869a7 00007f3c68e960bf 00007f3c68e96038
0709 12:19:17.789993 (+191443us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.029000 (+239007us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.042800 (+ 13800us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.042930 (+   130us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:18.042962 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.043029 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.554142 (+511113us) raft_consensus.cc:1685] finished
0709 12:19:18.554243 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.554368 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2135,"prepare.queue_time_us":134849,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":335,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":9476,"spinlock_wait_cycles":3255552,"thread_start_us":296956,"threads_started":2,"wal-append.queue_time_us":214397}
I20260709 12:19:18.557124 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.493762 24810 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:18.560079 24810 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.564527 18443 raft_consensus.cc:2468] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:18.565382 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.570384 18443 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:18.572118 18443 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:18.574441 18366 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:18.508247 24829 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:18.575845 24958 log.cc:927] Time spent T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:19:18.576076 24829 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.578449 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.496896 24937 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:18.579843 24937 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.580235 24937 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:18.493408 24801 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.585151 24801 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.492913 24932 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.587651 24932 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.587950 24707 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:18.543727 24906 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:18.589241 24736 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.589365 24906 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.589831 24906 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:18.590543 24736 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:18.590943 24736 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:18.542783 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:18.597199 18291 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.472301 24788 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.613271 18297 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.592521 24957 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:18.592991 24964 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:18.594094 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.628167 18442 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:18.629920 18297 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.598616 24900 raft_consensus.cc:2804] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:18.601547 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58849us
I20260709 12:19:18.540797 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.615197 24736 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.610919 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.473169 24795 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.612718 18443 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:18.646772 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.621850 24922 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:18.614619 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 100849us
I20260709 12:19:18.615442 19007 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:18.613701 24663 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.524s	user 0.002s	sys 0.007s
I20260709 12:19:18.654042 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:18.622133 24964 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:18.631891 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.636528 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.659626 19044 raft_consensus.cc:2393] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:18.633720 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.531821 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:18.617724 24946 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.667413 24795 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.520007 24626 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.670672 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.645730 18264 leader_election.cc:400] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 12 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13.
W20260709 12:19:18.688001 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49230.1us
I20260709 12:19:18.688850 24736 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.620944 24781 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.658037 18296 raft_consensus.cc:2410] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:19:18.594647 24803 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.686812 24913 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.692116 24904 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:18.692950 18292 raft_consensus.cc:1240] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:18.598088 24943 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.658042 24856 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:18.671458 24934 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:18.671693 24837 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.672394 24964 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:18.673662 24961 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 12:19:18.674364 24962 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:19:18.674749 24905 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:18.677490 24811 consensus_meta.cc:287] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.855s	user 0.009s	sys 0.000s
I20260709 12:19:18.678369 24896 raft_consensus.cc:3055] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:18.681528 24941 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.682922 24971 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.689090 24879 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:18.691007 24736 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.472656 24890 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:18.694748 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 79389.6us
I20260709 12:19:18.705569 24934 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:18.645675 19039 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:18.816926 24940 consensus_meta.cc:287] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.627s	user 0.004s	sys 0.011s
I20260709 12:19:18.737257 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:19:18.742254 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.759624 24833 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:18.760691 24943 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.760864 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:18.761492 24913 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.763528 24906 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.658385 24900 raft_consensus.cc:697] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:18.764822 19035 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
W20260709 12:19:18.766469 18183 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:18.769503 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.770006 24674 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.775667 24884 consensus_queue.cc:1059] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:19:18.737475 24970 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:18.781970 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.693048 24628 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:18.816869 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 118179us
I20260709 12:19:18.821213 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:18.720439 24945 raft_consensus.cc:2804] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:18.843529 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 364899us
I20260709 12:19:18.847831 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:18.695648 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 163928us
I20260709 12:19:18.776821 24803 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.779501 24946 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.600818 18448 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
W20260709 12:19:18.856035 18368 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.014s	sys 0.019s
W20260709 12:19:18.883973 18368 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.015s	sys 0.019s
I20260709 12:19:18.698693 24932 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.145s
I20260709 12:19:18.810940 24929 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.912319 24905 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.912729 24929 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.914266 18296 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:18.914793 24929 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:18.861248 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 100457us
I20260709 12:19:18.819391 24896 raft_consensus.cc:740] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:18.868115 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 79021.5us
I20260709 12:19:18.869279 19035 pending_rounds.cc:85] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 13
I20260709 12:19:18.916498 19035 pending_rounds.cc:107] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 12:19:18.916787 19035 raft_consensus.cc:2889] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.877297 18299 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:18.824906 24811 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.918548 24946 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.919422 24811 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:18.881470 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.882867 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.921619 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.868273 24953 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:18.890154 24934 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:18.901325 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.927210 24736 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:18.934331 24736 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:18.923593 19042 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:18.935336 24736 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.928498 24946 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.929111 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:18.939904 19036 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:18.940960 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.941685 19036 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:18.942437 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.929383 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.943748 19041 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 16: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
W20260709 12:19:18.876060 18183 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 109638us
I20260709 12:19:18.928776 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:18.903625 24973 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:18.944802 19039 raft_consensus.cc:2393] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:18.929643 24913 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:18.945950 24913 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:18.901135 24943 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:18.904860 24837 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:18.907385 24945 raft_consensus.cc:697] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:18.930445 24811 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:19:18.947249 24945 consensus_queue.cc:237] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.948561 24811 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:18.949020 24811 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:18.927042 19038 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:19:18.927652 24906 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.961971 24632 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.963379 24632 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.965143 18448 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:18.966441 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.963407 24626 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:18.967747 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.968348 18441 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:18.908094 24900 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.970434 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.908982 24890 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.848538 24971 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.971642 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:18.911875 24937 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:18.919842 24941 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:18.861724 24940 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:18.977113 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:18.981679 18447 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.983141 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:18.983892 18292 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.989051 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:18.991115 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.993592 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.995163 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:18.996542 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 201561us
I20260709 12:19:18.998661 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:18.999416 18367 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.054777 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.058244 18292 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:19.059708 18447 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
W20260709 12:19:19.062727 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 67673.2us
I20260709 12:19:19.065285 18338 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:19.066847 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 182525us
I20260709 12:19:19.069320 18367 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 13.
I20260709 12:19:19.071043 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:18.973677 24811 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.073199 19036 raft_consensus.cc:2368] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:18.967160 24674 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.076323 19009 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.079002 18368 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:19:19.066177 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:18.946796 24913 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:19.085127 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 114550us
W20260709 12:19:19.087056 18339 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 2.218s (SENT)
I20260709 12:19:19.087533 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.003261 24926 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:19.089121 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.078541 24836 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.998236 24906 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:19:19.091670 24795 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:19.098520 24937 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.099591 24884 raft_consensus.cc:2764] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:19:19.094081 24788 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:19.107993 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.079506 24967 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.112124 19006 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.093003 24707 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:19.114245 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.119503 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.110270 18441 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:19.119606 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:19.120319 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.082424 24933 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:19.113643 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:19.118999 18372 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:19.128924 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.084901 24685 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.115165 24795 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.136547 24795 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.136826 24940 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.139008 24856 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:19.119611 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.146512 18292 raft_consensus.cc:2393] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:19.108685 24970 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:19.122277 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:19.123778 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.123951 18296 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.122368 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.124194 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.116295 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.122826 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.151930 18370 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:19.124840 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.124969 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.152977 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.125140 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.153692 18370 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.155041 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:19.122975 19044 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:19.126511 18374 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 18. Ops: []
I20260709 12:19:19.156554 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.095981 24837 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:19:19.131428 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.095645 24926 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.131840 24202 raft_consensus.cc:2804] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.132503 18369 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.184237 24202 raft_consensus.cc:697] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.132750 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.186065 18375 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:19.117091 24861 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:19:19.120374 24836 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:19.188215 24836 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.133560 24896 consensus_queue.cc:260] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.119261 24811 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.124317 24941 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.158376 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.151404 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.159910 18300 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:19.160519 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.196990 18367 raft_consensus.cc:3060] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.171665 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:19.198259 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46944.2us
I20260709 12:19:19.174276 24926 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.148345 24971 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:19.120834 24967 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.169533 19038 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:19.171693 24837 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:19.156446 24803 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:19:19.183043 24970 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:19.185433 24202 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.114477 24525 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:19.194872 18370 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:19.196348 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.197094 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.150894 24856 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.171047 24685 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:19.215782 18368 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:19.206530 19037 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.201671 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.148288 24707 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:19:19.219058 18367 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
W20260709 12:19:19.213907 18339 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.706s (SENT)
I20260709 12:19:19.203055 24953 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:19.223206 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.224436 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.206758 24837 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:19.227257 18371 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:19.237299 18339 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.838s (SENT)
W20260709 12:19:19.239307 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 98686.6us
I20260709 12:19:19.242498 24795 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.236850 19009 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:19.246902 24989 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 12:19:19.247349 24953 raft_consensus.cc:2804] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:19.247860 24953 raft_consensus.cc:697] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.247911 24635 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.248780 24953 consensus_queue.cc:237] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.253010 24795 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:19.259126 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.259884 18441 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:19.260746 18447 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:19.266955 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.205401 24529 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:19:19.203812 24941 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.226922 24632 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.271687 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.281342 19037 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
W20260709 12:19:19.306216 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:19.307256 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.206840 24987 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:19:19.236250 24890 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.259725 24833 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.349157 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.235963 24685 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.240875 24970 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:19.348235 18442 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:19.365682 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.371812 18371 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:19.371616 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.351619 24906 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:19.374284 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 319572us
I20260709 12:19:19.355528 24890 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.374728 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.366181 24796 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:19.367673 24964 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.366796 24632 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.379611 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.377050 24926 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.377518 24906 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.405167 24906 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.387198 24632 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.407730 24632 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.410801 24967 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.411885 18442 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.377306 24976 raft_consensus.cc:2764] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:19.388624 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.388064 24685 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:19.389966 24993 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:19:19.393980 24941 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.370923 24856 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.429709 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.432482 24781 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:19:19.441536 24781 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.353749 24837 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.388104 24796 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:19.382869 24970 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.401752 24843 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:19.437440 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 48920.8us
I20260709 12:19:19.463651 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:19.484380 24829 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:19.486333 19038 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.452068 24890 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:19.484071 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 325756us
I20260709 12:19:19.487032 24890 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.430188 24976 consensus_queue.cc:1059] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.059s
I20260709 12:19:19.489459 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.489894 24781 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:19.484813 24884 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:19:19.424523 24926 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:19.486253 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2969) took 1128 ms. Trace:
I20260709 12:19:19.433095 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.491696 18440 rpcz_store.cc:276] 0709 12:19:18.357816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.493552 (+135736us) service_pool.cc:224] Handling call
0709 12:19:19.486082 (+992530us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b5400422f54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:19.486200 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648097536}
W20260709 12:19:19.492544 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 59538.4us
I20260709 12:19:19.491185 18373 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:19.492987 24788 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.494635 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:19.495534 24856 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:19.476665 24635 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.496179 24856 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.496443 24635 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.477907 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.498855 18294 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:19.498111 24896 raft_consensus.cc:3060] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:19.465422 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:19.484596 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 214867us
I20260709 12:19:19.478295 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.432622 24811 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:19:19.456337 24934 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.497225 24856 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.504128 19040 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:19.504312 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.448422 19037 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1329) took 1207 ms. Trace:
I20260709 12:19:19.490293 18292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:19.463057 24822 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:19:19.507592 18292 raft_consensus.cc:606] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:19.508041 18292 raft_consensus.cc:614] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.508345 24972 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.508817 24972 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.508944 24837 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.509687 24972 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.510637 24972 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.490357 24781 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:19.477468 18300 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.434177 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.499222 24635 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:19.512890 18375 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:19.521716 24826 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.522084 24826 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.522466 24826 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.526271 24957 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.528169 24957 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.529155 24957 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.530846 24957 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.536859 24630 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:19.536865 19043 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:19.542255 18442 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:19.543610 18338 leader_election.cc:400] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:19.544030 18338 leader_election.cc:403] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:19.544659 24822 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:19.545210 24805 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:19.544692 24630 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:19.546450 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.548640 18299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:19.549379 18299 raft_consensus.cc:606] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.428537 18368 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:19:19.549870 18299 raft_consensus.cc:614] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.555526 24630 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.558053 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.451287 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:19.444217 24943 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:19.513716 24843 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.513998 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:19.514989 25007 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:19.511746 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.463557 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.608139 18444 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:19.506697 19037 rpcz_store.cc:276] 0709 12:19:18.240884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.240995 (+   111us) service_pool.cc:224] Handling call
0709 12:19:19.448196 (+1207201us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b5400834fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:19.448332 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998490368}
I20260709 12:19:19.516109 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:19.608831 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50832.7us
W20260709 12:19:19.609297 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 70047.8us
I20260709 12:19:19.609429 18300 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:19.609692 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.516952 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.610754 18297 raft_consensus.cc:3060] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:19.611716 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97805.1us
I20260709 12:19:19.472124 24605 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.612510 18266 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:19.613021 24605 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:19.613869 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.564247 24796 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:19.565078 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.569410 18368 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:19.577220 19040 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:19.619535 19043 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:19.620656 24878 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.621199 24878 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.623811 24878 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:19.512463 24781 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.624956 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.625357 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.625996 18443 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:19.586390 18412 leader_election.cc:400] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
W20260709 12:19:19.606024 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 154839us
I20260709 12:19:19.628372 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.629428 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:19.614609 24826 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.558959 19038 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:19.632992 24644 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.633195 24953 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.633574 24953 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.633914 24953 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.615899 24805 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.634357 24805 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.634742 24805 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.639693 18300 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.639958 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.640879 18299 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:19.642059 24525 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:19.642735 24525 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.643316 24525 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:19.643025 24953 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.632030 24826 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:19.649837 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:19.650442 19038 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.651747 24953 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.653263 18299 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:19.616861 24605 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.616760 24987 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:19.657984 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44195.3us
I20260709 12:19:19.658186 24987 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:19.663923 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.664896 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.665221 24632 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:19.664955 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.665655 24632 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.665796 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.666106 24632 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.667062 24644 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:19.668236 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:19.668789 19040 raft_consensus.cc:606] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:19.664009 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 77837.2us
I20260709 12:19:19.670567 18371 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:19:19.671118 24674 consensus_queue.cc:1059] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:19.674429 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.675375 18441 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:19.669143 19040 raft_consensus.cc:614] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.681541 19040 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:19.682070 19040 raft_consensus.cc:606] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.682426 19040 raft_consensus.cc:614] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.685716 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.687903 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.688401 19038 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:19.689337 18445 raft_consensus.cc:3055] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:19.689719 18445 raft_consensus.cc:740] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.690534 18445 consensus_queue.cc:260] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.691676 18445 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.696693 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.695204 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:19.699666 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:19.700488 18300 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:19.702550 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.703220 18300 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.704566 18445 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:19:19.706468 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.707643 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:19.711220 18371 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.709231 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.708473 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.715061 19044 raft_consensus.cc:1240] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:19.695449 24934 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:19.693863 24970 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.721284 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.720430 18300 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:19.723892 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:19.773871 18296 raft_consensus.cc:1240] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.705193 24822 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.734663 18373 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:19.784377 18297 raft_consensus.cc:2468] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:19.692121 24972 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 12:19:19.785578 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 61044.6us
I20260709 12:19:19.705312 24626 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:19.789968 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.698189 25000 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
W20260709 12:19:19.795142 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 78735us
I20260709 12:19:19.692445 24943 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.797468 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.723672 24905 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.712901 24971 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:19:19.806066 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.721144 24934 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.717917 24810 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.722062 24837 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:19:19.718570 24994 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.708103 24967 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:19:19.788841 24937 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.790280 24856 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:19.800830 18371 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
W20260709 12:19:19.824371 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.801215 24630 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:19:19.812321 24934 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:19.799188 24973 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.816576 19038 raft_consensus.cc:606] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:19.855044 19038 raft_consensus.cc:614] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.828342 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.724268 24904 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:19:19.826884 24987 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:19.796233 24822 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.800822 24972 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:19:19.721628 24644 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:19:19.824795 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:19.870676 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45956.4us
I20260709 12:19:19.873183 18371 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:19.821357 24810 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:19.864928 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:19.809913 24674 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:19.823696 24905 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.819316 24994 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:19.800112 24626 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.823515 24856 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.809031 24843 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.899541 24843 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.865728 24836 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:19.903625 24836 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:19.905023 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 107604us
I20260709 12:19:19.897264 24525 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.906002 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:19.878496 24833 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.856649 24632 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.870702 24514 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:19.908974 18371 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.924482 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:19.878991 24805 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:19.917212 24943 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:19.910578 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 85170.5us
I20260709 12:19:19.923672 24905 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.903540 24685 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:19:19.909881 24837 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:19.880714 24940 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:19:19.911950 24878 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.894511 24972 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:19.931149 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:19.951275 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:19.951928 19043 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:19.952159 18443 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.894503 24674 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.936808 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.936946 19039 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:19.940718 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:19.941170 19041 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:19.959209 19042 raft_consensus.cc:2368] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:19.923132 18447 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:19.905741 24994 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.948897 18374 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:19.935523 19040 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:19.952420 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.968798 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.969327 18449 raft_consensus.cc:2376] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22. Re-sending same reply.
I20260709 12:19:19.932667 24833 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:19.953148 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.953222 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.953984 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.954671 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:19.924520 24514 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.959924 24973 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:19.964843 18295 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:19.970173 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:20.000415 18295 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:19.982812 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:20.001196 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:19.952423 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:20.002619 18300 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:19.957618 24856 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:19.980188 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:20.003125 18445 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:19.963702 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:19.991497 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:20.001816 18449 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:19.972121 18298 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.983603 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:19.979965 19039 raft_consensus.cc:1240] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:19.983062 18374 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:19:20.061846 18374 pending_rounds.cc:85] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 13
I20260709 12:19:20.062256 18374 pending_rounds.cc:107] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:19:20.062556 18374 raft_consensus.cc:2889] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.004108 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:20.061182 18296 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:20.066052 18443 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:20.061797 18369 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:20.069491 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:20.001453 18295 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:20.084880 18295 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.929188 24986 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:19:20.119323 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:20.087800 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 129453us
W20260709 12:19:20.101701 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.060384 24980 log.cc:927] Time spent T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:19:19.960009 24972 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:19.982510 24822 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:19.999931 24514 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:20.134765 19039 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.979750 24525 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:19.957937 24837 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:20.137533 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 393108us
I20260709 12:19:19.966591 24941 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 12:19:20.066397 24626 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:20.122671 24890 consensus_meta.cc:287] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.000s
I20260709 12:19:20.131145 18295 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:19:20.158493 18295 pending_rounds.cc:85] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Aborting all ops after (but not including) 9
I20260709 12:19:20.158916 18295 pending_rounds.cc:107] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:19:20.159282 18295 raft_consensus.cc:2889] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:20.137872 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:20.122200 24856 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:20.145375 18338 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:20.161453 18446 raft_consensus.cc:2410] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:19:20.161886 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:20.168301 18448 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.958099 24810 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.000417 24973 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:20.178953 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 245351us
I20260709 12:19:20.179023 18447 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:20.083087 18296 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:20.157373 24822 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:20.344676 24972 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.182643 18447 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:20.360375 24822 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.360817 24822 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:20.361718 18447 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:20.347023 24801 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.362977 18447 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:20.363072 24801 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:20.179767 24890 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:19:20.161441 25027 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 12:19:20.190981 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.346094 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:20.200140 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 130357us
I20260709 12:19:20.365096 18447 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:19:20.279572 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 208680us
I20260709 12:19:20.346851 19039 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:19:20.367558 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:20.240236 25033 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:19:20.348517 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 170213us
I20260709 12:19:20.180356 24833 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:20.357110 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.364712 24934 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.365394 24801 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.376652 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:20.377748 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.431174 24843 consensus_meta.cc:287] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.003s
I20260709 12:19:20.554577 18374 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:20.379166 19042 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:20.555637 18448 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
W20260709 12:19:20.429975 25020 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.455s	user 0.001s	sys 0.000s
I20260709 12:19:20.426807 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:20.403635 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.441478 19037 raft_consensus.cc:2368] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
W20260709 12:19:20.472996 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.567513 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.500027 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.423576 25032 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:19:20.347129 24884 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:20.526647 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.381588 24926 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.382126 24890 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:20.398079 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 210775us
I20260709 12:19:20.368899 18412 leader_election.cc:400] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:20.574499 24976 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:20.592367 18448 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:19:20.592573 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 223771us
I20260709 12:19:20.609468 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:20.612445 18447 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:20.619408 19008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.070s	sys 0.156s
W20260709 12:19:20.571137 25004 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:19:20.560091 25041 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.624475 19042 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:20.541837 24805 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.560446 24632 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.646216 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:20.614033 24626 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.575040 24884 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:20.650723 19008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.070s	sys 0.156s
I20260709 12:19:20.630563 25041 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.657002 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.631762 24943 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.633831 24890 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:20.636036 24674 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:20.644224 24685 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.654392 25038 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.686517 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:20.687353 18368 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:20.699342 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:20.658109 24626 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.713244 18368 raft_consensus.cc:2468] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:20.714811 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.661002 24632 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 12:19:20.717897 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 341285us
I20260709 12:19:20.721396 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:20.722905 18368 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.670415 24943 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.711328 24890 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.681514 24685 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.721982 19009 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:20.701501 24525 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:20.703594 24973 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.712826 24837 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.683514 24822 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.720880 24674 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:20.723537 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:20.710009 25040 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:19:20.766521 24836 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.767982 24884 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:20.768468 24514 consensus_meta.cc:287] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.000s
I20260709 12:19:20.768898 24514 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:20.769307 25041 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.688423 25038 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.729380 25042 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.748960 24626 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:20.731493 24994 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.000s
I20260709 12:19:20.750958 24822 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:20.734671 24843 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:20.734994 24934 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.735878 24833 raft_consensus.cc:2804] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:20.736645 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:20.808061 18291 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:20.737221 19039 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:20.737524 24525 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:20.738029 25030 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 12:19:20.740355 24837 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.756211 24685 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:20.722469 25000 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:19:20.740355 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:20.761734 24943 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:20.741257 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:20.764760 24905 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.829s	user 0.002s	sys 0.008s
I20260709 12:19:20.771549 18297 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:20.772990 18339 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:20.816439 18295 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 22: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:20.775349 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 95805us
I20260709 12:19:20.805212 24996 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.730944 24976 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:19:20.744148 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 85439.9us
I20260709 12:19:20.797379 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.822784 25000 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:20.822925 18369 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:20.823307 25000 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:20.823993 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.824615 18369 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:20.824193 25000 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.825378 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.802075 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39604 (request call id 55) took 1110 ms. Trace:
I20260709 12:19:20.825955 19038 rpcz_store.cc:276] 0709 12:19:19.691615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.693446 (+  1831us) service_pool.cc:224] Handling call
0709 12:19:20.801874 (+1108428us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e76853 00007f3c6de4460a 00007f3c673ca7d4 00007f3c673ca762 00007f3c673ca691 00007f3c673ca60c 00007f3c673c98c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:20.802007 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99244928}
I20260709 12:19:20.826310 24996 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.828239 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:20.829541 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:20.744870 25031 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:19:20.830165 18448 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:20.830889 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:20.727370 25033 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:20.832003 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:20.831853 25033 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.832705 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:20.743863 19040 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:20.829219 24996 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:20.833545 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:20.834363 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:20.817405 18299 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:20.741732 18368 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:20.835781 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:20.836241 18368 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:20.836745 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:20.837198 18368 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:20.818729 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45812.4us
I20260709 12:19:20.839936 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:20.827206 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:20.841153 18294 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:20.842186 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:20.811638 24514 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:20.845947 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:20.803870 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.802302 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 99815.4us
I20260709 12:19:20.825446 25000 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.850983 18368 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:20.812907 24663 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:20.813861 24833 raft_consensus.cc:697] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:20.855780 19035 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1345) took 1158 ms. Trace:
I20260709 12:19:20.854606 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:20.805400 24994 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:20.880889 25022 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.846246 24970 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:19:20.881533 24953 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:20.881949 24953 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.882303 24953 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:20.885221 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:20.886106 19035 rpcz_store.cc:276] 0709 12:19:19.697504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.715425 (+ 17921us) service_pool.cc:224] Handling call
0709 12:19:19.715766 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:19.716140 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:19.716171 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:19.717783 (+  1612us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.072919 (+355136us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:20.072964 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.073026 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:20.855559 (+782533us) raft_consensus.cc:1685] finished
0709 12:19:20.855578 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.855707 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":319,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4559,"thread_start_us":353863,"threads_started":1,"wal-append.queue_time_us":459795}
I20260709 12:19:20.881243 19009 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:20.888795 25022 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:20.889039 24934 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:20.889639 24934 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:20.890458 24934 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.891183 25022 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:20.893134 25022 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.893772 25022 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.896440 25022 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:20.897550 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:20.898547 25022 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:20.899158 25022 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:20.900218 25022 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.905715 24953 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:20.819860 24837 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:20.915592 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:20.985368 19038 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:20.948470 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 95510.2us
I20260709 12:19:20.819609 24905 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:20.802738 25042 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:20.953177 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 61334us
I20260709 12:19:20.879853 24685 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.003328 18442 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:20.950661 18183 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 177254018ba7448aa2b9110c9c9be9e3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:20.952397 24973 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.878350 24795 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:20.908957 24707 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:20.989511 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:21.023027 18446 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:21.023690 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.024048 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.024680 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.024825 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.001752 25042 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.002360 25056 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:21.010654 19038 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:21.022933 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:20.909096 25023 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:20.999305 25050 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:20.951038 24836 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:19:20.949447 24663 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.033260 18263 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:21.011482 24685 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.087845 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:21.089401 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 56255.6us
I20260709 12:19:21.011674 24900 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.086508 25023 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:21.098006 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.099524 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.117623 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1527) took 1615 ms. Trace:
I20260709 12:19:21.118672 19044 rpcz_store.cc:276] 0709 12:19:19.502714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.502836 (+   122us) service_pool.cc:224] Handling call
0709 12:19:19.503140 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.715590 (+212450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.117356 (+1401766us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540071e8d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:21.117560 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486838144}
I20260709 12:19:21.029235 24926 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.122606 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.129822 19038 raft_consensus.cc:2410] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:21.034518 25019 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.094426 24685 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.085970 25042 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.084589 24905 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.094554 24986 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:21.030022 24707 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:20.987758 25049 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.142s
I20260709 12:19:21.018218 24973 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.099092 25057 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.093833 24934 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.087934 25048 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.098706 25022 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.141860 24943 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.096176 25050 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.030411 24833 consensus_queue.cc:237] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.121289 24900 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.185755 25048 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.195744 25048 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.142375 25019 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.157403 24986 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:21.202514 24900 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.202845 24953 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.167073 24934 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.159307 25038 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.200081 24986 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.205845 24934 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:21.186748 25054 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:19:21.166594 25056 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:21.167848 24926 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:19:21.153553 24905 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.160719 25057 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.208202 24900 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.217211 24900 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.220055 25056 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.190441 24943 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.153825 25042 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.227032 25042 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.228122 25056 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.210834 25019 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.190325 25022 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.229986 25023 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.231986 25050 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:21.016537 18183 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67042.3us
W20260709 12:19:21.263592 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.123616 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:21.273155 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 149494us
W20260709 12:19:21.280092 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.286676 24514 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:21.288250 24514 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.292593 24514 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.186435 24525 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.294461 24514 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.310220 25063 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.310683 24795 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:21.312728 24707 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.220598 24900 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.313369 24795 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.315640 25056 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:21.316635 25056 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.317816 25075 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.328207 24890 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.329440 24890 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.331568 24926 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.317534 25056 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.349725 25056 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.336197 25042 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.339731 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.308550 24970 raft_consensus.cc:476] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Not starting pre-election -- already a leader
I20260709 12:19:21.341027 24663 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.341450 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.351015 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.352495 25050 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.352702 24890 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.333241 25022 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.332513 24926 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.311331 25063 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.355100 18447 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:21.318796 25075 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.318840 24811 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.366183 18446 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:21.366663 19036 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1536) took 1716 ms. Trace:
I20260709 12:19:21.367991 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:21.369306 19037 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 1145) took 1432 ms (client timeout 1859 ms). Trace:
W20260709 12:19:21.369968 19037 rpcz_store.cc:269] 0709 12:19:19.936461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.936605 (+   144us) service_pool.cc:224] Handling call
0709 12:19:21.368761 (+1432156us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540055e6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:21.369045 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109480192}
I20260709 12:19:21.370477 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.371130 24996 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.372349 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.372572 24996 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.367153 19036 rpcz_store.cc:276] 0709 12:19:19.649789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.650098 (+   309us) service_pool.cc:224] Handling call
0709 12:19:19.650393 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.816239 (+165846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:19.816288 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:19.816347 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:19.816404 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.366068 (+1549664us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540010c7d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:21.366150 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.366437 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380795648}
I20260709 12:19:21.373212 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.375675 25042 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.377334 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.378187 18447 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:21.378473 25042 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.379745 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2429) took 1408 ms. Trace:
I20260709 12:19:21.380473 19035 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:21.380291 18294 rpcz_store.cc:276] 0709 12:19:19.951746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.952193 (+   447us) service_pool.cc:224] Handling call
0709 12:19:19.952515 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.841582 (+889067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.339588 (+498006us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b540021ff54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:21.360168 (+ 20580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043301504}
I20260709 12:19:21.374505 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.385659 25063 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.381723 18444 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:21.382325 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.392251 24970 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.393178 24970 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.393949 25075 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.394793 25050 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.396400 24970 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.396540 24996 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.400760 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:21.371520 19041 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:21.398272 25050 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.385968 19038 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:21.363766 24811 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:21.414225 24957 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.415808 24796 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.427763 24796 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.429299 25063 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.416524 25050 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.417770 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:21.422181 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47392.1us
I20260709 12:19:21.381134 24926 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.443992 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.447904 24970 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.430122 25063 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.448314 24788 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.448717 24970 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.450157 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:21.451251 24970 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.451978 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:21.452180 24929 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:21.452499 18447 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.452747 24929 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.453347 24929 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:21.450834 25063 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:21.462728 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45153.6us
I20260709 12:19:21.426925 24943 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.445170 25050 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.472370 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:21.473850 18412 leader_election.cc:400] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22.
I20260709 12:19:21.475626 25042 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:21.477874 25042 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:21.478406 25042 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.430313 24514 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.486331 19038 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:21.478955 25042 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.490862 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.470086 25050 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.497306 24945 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:21.497819 24945 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.498245 24945 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.504042 24811 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:21.509151 25059 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.556s	user 0.000s	sys 0.001s
W20260709 12:19:21.512398 25068 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.520s	user 0.002s	sys 0.000s
W20260709 12:19:21.518465 18297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 2425) took 1567 ms (client timeout 1582 ms). Trace:
W20260709 12:19:21.518959 18297 rpcz_store.cc:269] 0709 12:19:19.950647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.953770 (+  3123us) service_pool.cc:224] Handling call
0709 12:19:21.517952 (+1564182us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54001cb6d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:21.518204 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767895424}
I20260709 12:19:21.519658 24896 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.520591 24905 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.520668 24896 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.507095 25057 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.526078 19041 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:21.529943 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.531746 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:21.533779 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.535604 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.536530 18443 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:21.538040 24781 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.543466 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.546137 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.546731 24896 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:21.547614 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 56055.9us
I20260709 12:19:21.506508 24811 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.536624 24943 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.540729 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.541249 24996 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:21.487654 24796 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:21.541527 25000 consensus_meta.cc:287] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.011s
I20260709 12:19:21.538986 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:21.505283 25053 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.517s	user 0.003s	sys 0.000s
I20260709 12:19:21.469990 24957 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.565166 24626 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.565922 24626 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.567278 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.567929 24896 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.568333 24626 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.574153 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.568892 24896 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.573151 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.583577 18367 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:21.584203 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.587519 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.587877 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.588634 18371 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:21.571218 24996 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:21.583451 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.603528 24822 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:21.607326 18370 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:21.607414 24904 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.608150 25056 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.608311 24904 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.608685 25056 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.590732 24996 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:21.610780 18442 consensus_meta.cc:287] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.607s	user 0.016s	sys 0.004s
I20260709 12:19:21.611328 18442 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:21.614008 24943 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.615546 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:21.617152 25043 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.889s	user 0.002s	sys 0.000s
I20260709 12:19:21.619413 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.620222 18292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2431) took 1662 ms. Trace:
I20260709 12:19:21.622232 18449 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:21.620631 18292 rpcz_store.cc:276] 0709 12:19:19.957986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.958143 (+   157us) service_pool.cc:224] Handling call
0709 12:19:19.958435 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.087725 (+129290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.087761 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.088236 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.536131 (+447895us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:20.677434 (+141303us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.677740 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.516497 (+838757us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b540004db54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:21.516572 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.616211 (+ 99639us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:21.619760 (+  3549us) raft_consensus.cc:1685] finished
0709 12:19:21.619785 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.620041 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125406,"prepare.queue_time_us":191184,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":119887,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":19636,"spinlock_wait_cycles":30156800,"thread_start_us":578700,"threads_started":2,"wal-append.queue_time_us":623852}
I20260709 12:19:21.622968 19035 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:21.623008 24876 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.623528 18366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2774) took 1678 ms. Trace:
I20260709 12:19:21.607510 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:21.626500 24943 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.629338 18374 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:21.630707 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.631973 18372 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:21.628592 18366 rpcz_store.cc:276] 0709 12:19:19.944875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.944986 (+   111us) service_pool.cc:224] Handling call
0709 12:19:19.971462 (+ 26476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.346583 (+375121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.346612 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.346653 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:20.346688 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.622874 (+1276186us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54000f8a54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:21.622957 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.623190 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862124800}
I20260709 12:19:21.618522 24685 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.642810 18375 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55012 (request call id 2765) took 1973 ms. Trace:
I20260709 12:19:21.637696 24943 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:19:21.644313 25089 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:21.652859 24943 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.640270 24945 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:21.664038 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:21.664410 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.656777 24904 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.644585 18375 rpcz_store.cc:276] 0709 12:19:19.661677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.663067 (+  1390us) service_pool.cc:224] Handling call
0709 12:19:19.663422 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:21.344883 (+1681461us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18004d0fe0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c68a244bc 00007f3c65d7248b 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e74bfb 00007f3c68e7f64b
0709 12:19:21.345328 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.345370 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:21.346741 (+  1371us) log.cc:844] Serialized 52 byte log entry
0709 12:19:21.441197 (+ 94456us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:21.447539 (+  6342us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.447875 (+   336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.448047 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.629114 (+181067us) raft_consensus.cc:1685] finished
0709 12:19:21.634635 (+  5521us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.634820 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45601,"prepare.queue_time_us":3901,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":64662,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":10291,"spinlock_wait_cycles":5096704,"thread_start_us":166343,"threads_started":2,"wal-append.queue_time_us":112254}
I20260709 12:19:21.663057 24943 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.682766 18366 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:21.684791 24876 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.687038 18372 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:21.671375 24811 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.680665 24795 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.702364 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2837) took 1314 ms (client timeout 1612 ms). Trace:
I20260709 12:19:21.702739 24795 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.707449 24933 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.708289 24933 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.703215 24896 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.712332 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.748523 24933 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.749099 18445 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:21.750294 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.714619 24795 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.718683 25000 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.756333 25000 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.758713 25000 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:21.760383 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 56069.5us
I20260709 12:19:21.724045 24983 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:21.763799 24983 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.765100 24987 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:21.766090 24983 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.765861 24987 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.766631 25042 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.724718 24985 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:21.724473 24943 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.768313 18292 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:21.768107 24985 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.769335 24943 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.731588 24514 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.733858 18373 raft_consensus.cc:1240] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:19:21.735826 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.736091 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1153) took 1068 ms. Trace:
I20260709 12:19:21.736650 24811 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.737478 24685 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:21.702919 18442 rpcz_store.cc:269] 0709 12:19:20.387775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.426529 (+ 38754us) service_pool.cc:224] Handling call
0709 12:19:21.699894 (+1273365us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400423e54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:21.702282 (+  2388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007517568}
I20260709 12:19:21.739745 24994 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.742528 25056 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.744264 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1527) took 1106 ms. Trace:
I20260709 12:19:21.755223 25051 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.785801 25051 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.786243 24973 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.788141 25051 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.789487 24811 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.790603 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.791579 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.794028 19044 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:21.783030 18370 rpcz_store.cc:276] 0709 12:19:20.638002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.638259 (+   257us) service_pool.cc:224] Handling call
0709 12:19:21.744017 (+1105758us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400395154. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:21.744171 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183153408}
I20260709 12:19:21.802248 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.795208 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.769629 25042 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.806114 19040 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 22: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:21.806023 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:21.806479 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.806758 18366 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.807710 18297 raft_consensus.cc:2410] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:21.795646 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.814934 18373 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:21.776300 19042 rpcz_store.cc:276] 0709 12:19:20.667629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.667763 (+   134us) service_pool.cc:224] Handling call
0709 12:19:20.668098 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.679904 (+ 11806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.679948 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.685183 (+  5235us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.803954 (+118771us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:20.803997 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.804061 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.504306 (+700245us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:21.735559 (+231253us) raft_consensus.cc:1685] finished
0709 12:19:21.735582 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.735922 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11345,"raft.queue_time_us":11974,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":16699,"spinlock_wait_cycles":24266496,"thread_start_us":77587,"threads_started":1,"wal-append.queue_time_us":147254}
I20260709 12:19:21.816233 24876 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.816597 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.817274 18375 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:21.819427 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.820279 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.815006 18366 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:21.797533 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:21.823594 18447 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:21.835796 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.837966 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.840706 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:21.845180 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.845950 18297 raft_consensus.cc:2468] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:21.846864 25042 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.847640 25042 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:21.858783 18444 raft_consensus.cc:3055] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:21.859386 18444 raft_consensus.cc:740] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:21.863058 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.863255 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:21.866825 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.868322 24987 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.868904 24822 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:21.869436 24822 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:21.871989 18444 consensus_queue.cc:260] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.872715 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:21.873390 18444 raft_consensus.cc:3060] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.873682 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.874934 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.875641 18368 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:21.876536 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.877813 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:21.877944 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.878306 18368 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.878741 19040 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:21.879825 24957 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.876456 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.866364 24985 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:21.880970 18341 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.873401 18369 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:21.869444 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 72068.9us
I20260709 12:19:21.883687 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.885113 24514 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:21.887576 24514 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.887912 24514 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.892344 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.892761 24972 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:21.897945 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:21.898358 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.899121 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.899807 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.900739 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.903062 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.884351 24929 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.909310 24929 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.910189 18295 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:21.924188 25076 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.926949 24957 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:21.926820 25076 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.927716 24957 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:21.928504 25044 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 1.206s	user 0.004s	sys 0.000s
I20260709 12:19:21.929203 25076 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.930155 24957 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.921283 25042 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:21.931628 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.933929 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.922961 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:21.934577 18299 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:21.935308 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:21.936086 18291 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.937704 25042 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:21.938325 25042 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.940455 25042 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:21.942960 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.943670 18441 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:21.944934 24945 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.945004 25050 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.952105 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.953964 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:21.954370 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.954515 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.954829 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.955183 18372 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:21.956255 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.957237 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:21.957630 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.960788 18369 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:21.960871 25052 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.961731 25052 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:21.961205 18295 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:21.962297 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.968628 25052 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:21.968820 25076 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:21.969341 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:21.987190 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.970610 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.971524 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.988601 18370 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:21.989349 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.989713 18370 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.990209 18375 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:21.973474 18366 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:21.974225 18369 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:21.974695 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.971421 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.991354 18441 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.992165 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.993043 18294 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:21.993970 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.978101 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.979034 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.979187 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.981020 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.981119 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:21.970532 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.994578 18339 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:21.998389 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.998565 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:21.981140 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:21.999094 18293 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.982471 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.000142 18443 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:21.984267 18263 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:21.985603 18444 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:19:21.987717 24985 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.975787 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.992214 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:21.976778 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:21.977707 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:21.977425 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:21.978688 24822 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.019487 18367 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.019743 18447 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:22.249936 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.158771 18441 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
W20260709 12:19:22.168123 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 44716.9us
W20260709 12:19:22.135473 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 154441us
I20260709 12:19:22.078898 18300 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:22.180639 18293 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:22.025413 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.251739 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.056269 18291 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:22.018834 18445 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:22.017807 18444 pending_rounds.cc:85] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 13
I20260709 12:19:22.077339 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:22.303874 18368 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:22.134997 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:22.075556 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:22.283380 25050 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:22.323097 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.048044 24987 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.135890 24913 raft_consensus.cc:2764] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:19:22.222899 24994 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:22.307448 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.331634 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:22.334568 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 329750us
I20260709 12:19:22.048307 25025 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.336494 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.281153 18444 pending_rounds.cc:107] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:19:22.344046 18444 raft_consensus.cc:2889] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.320489 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:22.337767 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.339630 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.340752 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.025722 25057 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:22.345494 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:22.346045 18291 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:22.132865 24943 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.205s
I20260709 12:19:22.347451 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:22.344871 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.355597 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.359408 18370 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:22.131568 24514 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:22.351199 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:22.337054 24987 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:22.345043 25056 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:22.372557 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:22.370594 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:22.322764 24876 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:19:22.383541 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:22.384074 19042 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.386485 18299 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:22.372409 25056 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:22.391227 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.371683 24987 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.406432 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:22.370059 24994 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.411213 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.411996 18441 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:22.413542 18298 raft_consensus.cc:2393] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:22.386412 24822 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:22.390731 25056 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:22.423070 18446 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:19:22.424894 18300 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:22.428417 19040 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:19:22.429203 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 294460us
I20260709 12:19:22.407768 25050 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:22.428385 18446 pending_rounds.cc:85] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 13
I20260709 12:19:22.413061 24943 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:22.440559 18446 pending_rounds.cc:107] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:19:22.443305 18446 raft_consensus.cc:2889] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:22.438284 18412 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.65:37771 timed out after 1.582s (SENT) [suppressed 20 similar messages]
I20260709 12:19:22.459065 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:22.463382 25019 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:22.464079 25019 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:22.464751 24994 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:22.466301 25019 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:22.466876 18367 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:22.438655 25050 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:22.468043 18447 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:22.470795 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:22.483511 18443 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:22.484357 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.417158 25057 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:22.486768 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 50485.3us
I20260709 12:19:22.489710 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:22.469516 24994 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.480655 25050 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:22.471305 24943 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:22.501515 19009 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:22.503566 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.494766 24685 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:22.510099 24945 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:22.516137 18300 raft_consensus.cc:1275] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:22.516256 24945 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:22.517292 24945 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.518263 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 3001) took 1231 ms. Trace:
I20260709 12:19:22.497993 25042 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.530058 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:22.530834 18368 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:22.532927 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:22.548593 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:22.554035 18374 raft_consensus.cc:1275] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:22.555994 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:22.525652 24685 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
W20260709 12:19:22.547446 24972 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.003s
I20260709 12:19:22.565970 18370 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:22.563918 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:22.566883 18299 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 14.
I20260709 12:19:22.538122 24943 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.518721 18440 rpcz_store.cc:276] 0709 12:19:21.299101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.301363 (+  2262us) service_pool.cc:224] Handling call
0709 12:19:21.364202 (+ 62839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.423064 (+ 58862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.423115 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.825343 (+402228us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:19:21.852628 (+ 27285us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.324058 (+471430us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:22.324096 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.324170 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.517850 (+193680us) raft_consensus.cc:1685] finished
0709 12:19:22.517870 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.518077 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4720,"raft.queue_time_us":25313,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":2930,"spinlock_wait_cycles":43382656,"thread_start_us":440379,"threads_started":1,"wal-append.queue_time_us":633713}
I20260709 12:19:22.580464 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:22.565970 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:22.581858 18448 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:22.582669 25057 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:22.589380 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:22.590180 24685 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.595846 19041 raft_consensus.cc:1240] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:22.578738 25101 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:19:22.611799 18447 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:22.564368 25042 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:22.635993 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 2995) took 1613 ms. Trace:
I20260709 12:19:22.636550 18448 rpcz_store.cc:276] 0709 12:19:21.022604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.022721 (+   117us) service_pool.cc:224] Handling call
0709 12:19:22.635767 (+1613046us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400376254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:22.635930 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3559044864}
I20260709 12:19:22.636937 18264 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:22.637696 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:22.645538 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:22.658063 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:22.656916 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:22.675674 24996 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:22.676000 24803 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:19:22.654367 24985 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:22.688712 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:22.672720 25076 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.738317 18298 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:22.694856 25076 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:22.757448 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 410247us
W20260709 12:19:22.776877 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 433267us
I20260709 12:19:22.757783 25076 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.799602 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:22.806414 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 368146us
W20260709 12:19:22.819849 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.820370 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.854146 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.806761 25019 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.856050 19042 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:22.857960 24957 raft_consensus.cc:2764] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:19:22.868227 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 77306.3us
I20260709 12:19:22.802423 24994 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:19:22.857868 25019 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:22.871778 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 234132us
I20260709 12:19:22.807384 24983 consensus_queue.cc:1059] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:19:22.863297 25042 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:22.879315 25019 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.883715 24985 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.906724 25029 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.941915 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 265099us
W20260709 12:19:22.938133 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 348830us
W20260709 12:19:22.943945 24987 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.003s
I20260709 12:19:22.948159 24929 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:22.950004 25106 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
W20260709 12:19:22.963750 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44574.5us
W20260709 12:19:22.954974 25113 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.171s	user 0.001s	sys 0.001s
I20260709 12:19:22.982831 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:22.950937 24983 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 12:19:22.968459 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 180381us
I20260709 12:19:22.988794 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:22.955008 25057 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:19:22.993489 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2863) took 1020 ms. Trace:
I20260709 12:19:22.950912 24985 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:22.982537 24943 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:22.959580 24987 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.028677 25117 raft_consensus.cc:3055] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:23.012887 18296 rpcz_store.cc:276] 0709 12:19:21.972363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.974591 (+  2228us) service_pool.cc:224] Handling call
0709 12:19:21.974877 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.975176 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:22.334879 (+359703us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.965435 (+630556us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.965964 (+   529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.966087 (+   123us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:22.990596 (+ 24509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.991349 (+   753us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.992994 (+  1645us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796,"prepare.queue_time_us":42644,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":88,"raft.queue_time_us":13375,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4203,"thread_start_us":918382,"threads_started":3,"wal-append.queue_time_us":392607}
W20260709 12:19:23.034969 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 62675.9us
I20260709 12:19:23.038527 24985 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.039443 18294 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:23.040328 24987 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.016534 24941 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.973232 25029 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.052110 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2860) took 1092 ms. Trace:
W20260709 12:19:23.054277 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 70576.5us
I20260709 12:19:23.054168 18297 rpcz_store.cc:276] 0709 12:19:21.959918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.963007 (+  3089us) service_pool.cc:224] Handling call
0709 12:19:21.963490 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.963952 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.963980 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.964585 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.339968 (+375383us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:22.340033 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.340154 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.051827 (+711673us) raft_consensus.cc:1685] finished
0709 12:19:23.051858 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.052044 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5276,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":373746,"threads_started":1,"wal-append.queue_time_us":582875}
I20260709 12:19:23.021315 25042 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:23.060066 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 178789us
I20260709 12:19:23.027674 24943 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.032323 25117 raft_consensus.cc:740] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:23.017077 25111 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.214s
W20260709 12:19:23.104916 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 247312us
I20260709 12:19:23.091840 24943 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:23.018366 25110 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:19:23.105026 25125 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:23.111891 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.120981 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:23.117882 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 243097us
I20260709 12:19:23.131083 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:23.138208 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 101156us
I20260709 12:19:23.139636 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.131081 25042 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:23.145515 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 50352.9us
I20260709 12:19:23.146040 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.154917 18338 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.137928 24514 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.137655 25029 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:23.162948 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 215705us
I20260709 12:19:23.175372 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:23.145088 25121 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:23.201159 24913 raft_consensus.cc:2764] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:19:23.215902 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 61101.3us
I20260709 12:19:23.146121 24987 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:23.229391 25140 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 12:19:23.230446 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 55157.9us
I20260709 12:19:23.232069 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:23.233297 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.239678 25148 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.177366 25057 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:23.240449 25148 consensus_queue.cc:1048] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:23.209098 25147 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:23.263019 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 142124us
I20260709 12:19:23.266010 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.264667 25135 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.227762 24941 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:19:23.290793 18412 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:23.290165 25115 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.257336 25029 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:23.298681 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 61155us
I20260709 12:19:23.246222 25145 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.302883 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.303597 18369 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 13.
I20260709 12:19:23.306507 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.307109 25151 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:23.307518 18373 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:23.307685 25151 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:23.308646 25151 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.309177 25155 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:23.309923 25114 log.cc:927] Time spent T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:19:23.310425 25155 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:23.311101 25122 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 12:19:23.312458 25056 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.855s	user 0.012s	sys 0.001s
W20260709 12:19:23.312633 25120 log.cc:927] Time spent T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.284s	user 0.002s	sys 0.000s
I20260709 12:19:23.312906 25056 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:23.313372 25155 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.243882 25134 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:19:23.319881 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:23.322507 25118 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:19:23.219689 25121 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.324466 25128 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.328390 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.330147 25128 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.332391 25128 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.332659 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:23.333083 25127 log.cc:927] Time spent T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:19:23.277130 24987 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:23.328454 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.344245 18370 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:23.303213 24933 raft_consensus.cc:2764] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:19:23.347743 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.291647 25042 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:19:23.348632 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
W20260709 12:19:23.323179 25050 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.003s
W20260709 12:19:23.324944 25137 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:19:23.218755 25117 consensus_queue.cc:260] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:23.341884 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 76149us
I20260709 12:19:23.365160 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:23.366361 18370 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:23.374037 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:23.382045 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 225679us
W20260709 12:19:23.384181 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 202543us
I20260709 12:19:23.360224 25050 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.405071 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:23.415858 18440 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:23.421082 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:23.426769 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.386531 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:23.386389 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 212200us
I20260709 12:19:23.430363 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:23.431059 18447 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:23.432619 25117 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:23.433432 25121 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:23.434762 18182 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:23.416960 25134 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.435880 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.436313 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.436259 25134 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.438349 25134 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.438925 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.396441 24987 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.439739 18294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43604 (request call id 1288) took 1104 ms. Trace:
W20260709 12:19:23.440922 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54585.9us
I20260709 12:19:23.433466 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.442052 18291 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:23.443346 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:23.432631 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.444792 18299 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:23.440246 18294 rpcz_store.cc:276] 0709 12:19:22.335668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.340493 (+  4825us) service_pool.cc:224] Handling call
0709 12:19:23.439564 (+1099071us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54000aa0d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:23.439692 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594481024}
I20260709 12:19:23.446053 18295 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:23.432000 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141268us
W20260709 12:19:23.426451 18338 connection.cc:256] RPC call timeout handler was delayed by -1.1463004180000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:19:23.448865 18338 connection.cc:256] RPC call timeout handler was delayed by -1.0941886959999465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 12:19:23.467545 18295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2871) took 1484 ms. Trace:
I20260709 12:19:23.468017 18295 rpcz_store.cc:276] 0709 12:19:21.983176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.336646 (+353470us) service_pool.cc:224] Handling call
0709 12:19:23.440899 (+1104253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.447637 (+  6738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.467333 (+ 19696us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000aa0d0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:23.467481 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501998976}
I20260709 12:19:23.469856 25154 consensus_queue.cc:1048] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:23.446097 24941 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:23.471498 25154 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:23.458374 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:23.475464 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 90705.4us
I20260709 12:19:23.476121 18445 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:23.390694 25019 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.385149 25169 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:19:23.481544 18368 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:23.482496 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.483135 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.496307 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:23.497588 18375 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:23.499758 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.503840 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:23.505235 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.505857 19043 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:23.507355 25112 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.507504 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.508080 18370 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:23.508387 19035 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:23.500929 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:23.474776 25056 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.511649 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.511159 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.517468 19041 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:23.481426 25128 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:23.529057 25160 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:23.543047 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 122067us
I20260709 12:19:23.548290 19006 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:23.478878 25151 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:19:23.537024 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:23.552438 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.555819 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.556977 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:23.566299 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71006.3us
I20260709 12:19:23.543560 25029 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.530329 25159 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:23.572919 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:23.545480 25128 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:23.574782 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:23.590322 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.597182 24514 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.598670 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.598839 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.619549 24878 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.619477 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.619452 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.614717 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.571489 24933 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.599340 25057 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 12:19:23.639253 25174 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:19:23.639881 25141 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.640753 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:23.641166 24943 consensus_meta.cc:287] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.510s	user 0.004s	sys 0.004s
I20260709 12:19:23.641644 24943 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.642768 19043 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:19:23.643009 25167 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.642127 18415 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:23.624087 25171 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:19:23.594347 25050 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:23.627808 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 72748.2us
I20260709 12:19:23.599568 18296 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:23.643491 25019 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.627321 25149 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.546509 25025 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:19:23.581751 25154 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.586292 24822 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.595924 25128 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:23.685772 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 43736.3us
I20260709 12:19:23.746094 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:23.664543 25076 consensus_meta.cc:287] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.866s	user 0.011s	sys 0.002s
I20260709 12:19:23.685181 18291 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:23.696739 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:23.749655 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52991.2us
I20260709 12:19:23.666471 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.554746 25160 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.689038 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.644062 25143 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:23.547061 25112 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.671840 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.676971 18440 raft_consensus.cc:2410] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:19:23.667351 25116 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.587194 25125 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
W20260709 12:19:23.725409 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 79965.8us
I20260709 12:19:23.727180 24943 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.617038 25159 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.691102 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.650930 25167 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.679008 25146 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.754967 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.743722 24822 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.694264 25149 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:23.773388 18415 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:23.753808 25131 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:23.753085 25145 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.780337 18295 raft_consensus.cc:2393] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:23.756639 24994 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:23.796027 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.754998 25076 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.785487 25141 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:23.803712 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53171.7us
I20260709 12:19:23.783036 25025 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:23.788630 24822 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.786157 25146 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.819985 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.799443 24994 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.821082 24994 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:23.822271 25112 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.801416 25155 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:23.836783 19042 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:19:23.860302 19036 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:23.832245 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:23.838788 18338 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:23.849248 25179 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:19:23.861783 18294 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 22: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:23.835795 25025 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:23.801633 25145 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.856086 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.849534 25177 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.827562 25151 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.839310 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:23.862591 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.819159 25167 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:23.771329 25143 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.836969 24945 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:23.828189 25146 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.892639 25149 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.820442 25057 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.896365 25160 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:23.887706 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.868273 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:23.872059 25042 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:23.863559 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:23.862125 25169 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.910072 25169 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:23.912199 25169 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:23.910960 24983 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:19:23.901347 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:23.874782 19035 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 22.
I20260709 12:19:23.901865 25057 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:23.906215 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:23.906936 25187 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:23.908329 19036 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:23.909554 25076 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:23.915170 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 96538.2us
I20260709 12:19:23.922219 18294 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:23.919613 25057 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.925457 25057 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.926067 25187 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.928524 25187 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:23.929684 25176 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:19:23.930603 25187 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:23.899883 25150 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.931191 25159 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.931123 25187 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.918007 25042 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:23.932885 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 358182us
I20260709 12:19:23.933199 25187 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:23.935390 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.935848 18447 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:19:23.936385 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 97169.5us
I20260709 12:19:23.929546 25025 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:23.932581 25042 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.948751 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:23.922019 25151 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:23.948170 25150 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.899413 25146 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.961608 18264 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:23.967676 18442 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:23.969049 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:23.970579 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:23.971724 25191 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:23.971951 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:23.973066 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 64568.3us
I20260709 12:19:23.903614 24945 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.898288 25167 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:23.989511 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:23.990154 18442 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:23.942503 25160 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:19:23.956167 25151 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:23.997932 25194 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.992677 18338 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:23.975051 24878 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:19:23.998512 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 79310.2us
I20260709 12:19:24.017941 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.006196 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:24.026424 18442 raft_consensus.cc:2468] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:24.017702 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:23.964219 24994 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.033726 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.037834 18296 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:24.016757 25146 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.039238 25191 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:24.039695 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.002669 25151 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:24.040870 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53472us
I20260709 12:19:23.982852 25057 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:19:24.107811 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:24.089761 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.102244 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:24.112248 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:24.104033 25195 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:19:24.104399 25019 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:24.105751 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:24.110141 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:24.006522 25150 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.111713 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:24.106514 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 73333us
I20260709 12:19:24.017880 25167 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:24.116987 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110896us
I20260709 12:19:24.117493 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:24.118534 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:24.119689 19036 raft_consensus.cc:2393] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 23: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:24.121414 18442 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.991592 25193 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 12:19:24.018364 24943 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:19:24.033317 25149 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:19:24.142306 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:24.148806 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.027334 25194 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.113801 25056 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:19:24.159152 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:24.138490 25019 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:24.189296 18442 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:24.151898 25150 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:24.207664 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:24.184895 25190 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:24.213795 18296 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:24.190599 25112 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:24.201288 25025 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:24.204540 25056 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.204257 24878 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:19:24.220242 24933 consensus_meta.cc:287] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.003s
I20260709 12:19:24.213841 25190 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.201819 25128 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.226922 24878 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.220503 25025 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:24.227046 25056 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:24.236523 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 77480.3us
W20260709 12:19:24.240599 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 128420us
I20260709 12:19:24.231321 25190 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.231760 24933 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.237690 24878 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:24.253260 18299 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:24.241389 25056 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.228559 25112 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:24.262147 25192 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:19:24.262569 24933 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:24.242321 25172 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:24.277060 25172 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.280000 25128 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:24.252521 24994 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:24.290675 25145 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.292402 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:24.293661 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:24.249083 25149 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:19:24.294642 18338 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.278367 25056 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.283124 18446 raft_consensus.cc:1240] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:19:24.293686 25150 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:24.300593 25155 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:24.300884 24994 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.312346 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.314097 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.314996 18375 raft_consensus.cc:2410] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:24.316227 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.318482 18375 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:19:24.318895 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.242383 25019 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.320609 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.321345 18367 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:24.321734 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.322373 18374 raft_consensus.cc:2376] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17. Re-sending same reply.
I20260709 12:19:24.320567 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.307595 25150 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:24.310645 25155 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.343487 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:24.343843 25194 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.347559 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.304744 25149 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:19:24.350291 19037 raft_consensus.cc:2410] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:24.351572 18415 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:24.348385 18374 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:24.353016 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.329985 25205 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:24.368120 25162 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:24.368696 25162 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.338227 25150 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:24.405364 25183 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.473s	user 0.000s	sys 0.002s
I20260709 12:19:24.406028 25162 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:24.406410 25042 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.389843 25149 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:24.447805 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:24.339633 25155 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:24.396013 25205 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.411016 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:24.411509 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2875) took 1007 ms. Trace:
I20260709 12:19:24.416042 25131 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:24.416204 24822 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.007s
I20260709 12:19:24.343516 25112 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.418396 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:24.419414 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.420320 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.421599 18292 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:24.423836 25172 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.426407 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.428231 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.430112 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.434069 25056 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:24.479830 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 128322us
I20260709 12:19:24.438665 25151 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.439265 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.442354 24945 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:24.445572 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 355898us
I20260709 12:19:24.446789 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:24.455263 18441 rpcz_store.cc:276] 0709 12:19:23.403992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.404109 (+   117us) service_pool.cc:224] Handling call
0709 12:19:23.432927 (+ 28818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.494569 (+ 61642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.494763 (+   194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.530751 (+ 35988us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.794857 (+264106us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:23.794895 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.794957 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.404759 (+609802us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:24.411210 (+  6451us) raft_consensus.cc:1685] finished
0709 12:19:24.411244 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.411424 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15252,"raft.queue_time_us":106922,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":22482,"spinlock_wait_cycles":13165952,"thread_start_us":257599,"threads_started":2,"wal-append.queue_time_us":367779}
I20260709 12:19:24.484061 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.460747 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:24.478333 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.484740 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.481843 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.483420 18445 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:24.487679 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.446334 25202 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:24.462397 25131 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.495505 25220 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.507332 18444 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:24.497171 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.513278 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.518043 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:24.482630 25112 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:24.482838 25149 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.525753 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:24.527787 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:24.531975 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:24.533318 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48648.2us
I20260709 12:19:24.535218 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:24.520987 25135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.002s	sys 0.006s
W20260709 12:19:24.543592 25135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.002s	sys 0.006s
W20260709 12:19:24.545192 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 192236us
W20260709 12:19:24.547120 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69640us
I20260709 12:19:24.478914 24822 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.550257 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:24.555608 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.522953 25194 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.529146 25170 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.529328 25172 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.574898 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.581100 18338 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.550529 25177 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:24.596352 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:24.597524 19035 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:24.557533 25215 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:24.520583 25220 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.538362 25149 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.595012 25169 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.598032 25194 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.598961 25177 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:24.570812 25193 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:19:24.555377 25112 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:24.681638 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 100638us
I20260709 12:19:24.692512 18372 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:19:24.702921 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 159572us
I20260709 12:19:24.616667 25222 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:24.619490 25218 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:24.709844 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:24.618299 25056 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:19:24.718318 19038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59430 (request call id 1365) took 1222 ms. Trace:
I20260709 12:19:24.718830 19038 rpcz_store.cc:276] 0709 12:19:23.495783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.495948 (+   165us) service_pool.cc:224] Handling call
0709 12:19:23.529945 (+ 33997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.801648 (+271703us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:23.826756 (+ 25108us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.956115 (+129359us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.956168 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:23.956216 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.718015 (+761799us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400808254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:24.718079 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.718242 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25369,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":85,"raft.queue_time_us":15543,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":624106368,"thread_start_us":128835,"threads_started":1,"wal-append.queue_time_us":195529}
I20260709 12:19:24.726001 25219 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:19:24.731657 18266 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:24.695474 25177 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:24.739735 25042 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:24.739982 25205 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.740994 25190 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.742291 25220 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.743218 25151 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:24.744001 24822 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:24.715097 25218 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:24.747368 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 151079us
I20260709 12:19:24.748313 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:24.711884 25222 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:24.832530 18297 raft_consensus.cc:2368] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:24.834654 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.707906 25221 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:24.838121 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 89909.6us
I20260709 12:19:24.698359 25170 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 12:19:24.831964 25128 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.836259 25042 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:24.834303 25226 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.832370 25222 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:24.854346 18446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.005s	sys 0.005s
I20260709 12:19:24.833047 25234 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:19:24.846621 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 136842us
I20260709 12:19:24.838852 25193 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:24.843628 25212 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.847483 25173 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.864248 18300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43604 (request call id 1303) took 1033 ms. Trace:
W20260709 12:19:24.864053 18446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.006s	sys 0.005s
I20260709 12:19:24.871181 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:24.857128 25042 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.867552 25234 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:24.895450 25181 log.cc:927] Time spent T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.964s	user 0.000s	sys 0.005s
I20260709 12:19:24.903168 25042 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.904973 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.869395 25221 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:24.879359 18300 rpcz_store.cc:276] 0709 12:19:23.830922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.831077 (+   155us) service_pool.cc:224] Handling call
0709 12:19:23.866955 (+ 35878us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:23.879899 (+ 12944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.879948 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:23.902255 (+ 22307us) log.cc:844] Serialized 52 byte log entry
0709 12:19:23.981913 (+ 79658us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:19:23.982301 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 12:19:23.983329 (+  1028us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.454365 (+471036us) spinlock_profiling.cc:243] Waited 941 us on lock 0x7b18004c24c0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8c455 00007f3c68e74060 00007f3c68e86716 00007f3c68e97287 00007f3c68e97208
0709 12:19:24.856566 (+402201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.856639 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.864034 (+  7395us) raft_consensus.cc:1685] finished
0709 12:19:24.864058 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.864193 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":309,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":50482,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":189,"spinlock_wait_cycles":15538816,"thread_start_us":440849,"threads_started":2,"wal-append.queue_time_us":166758}
W20260709 12:19:24.944868 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 80131.9us
I20260709 12:19:24.960089 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:24.954648 24878 consensus_meta.cc:287] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.001s
I20260709 12:19:24.983278 24878 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.988914 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:24.990118 18373 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.993523 18370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1563) took 1441 ms. Trace:
I20260709 12:19:24.996116 25222 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.999459 25117 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.000133 25117 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:24.878742 25155 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.002331 18374 raft_consensus.cc:2368] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23.
I20260709 12:19:25.005272 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.006160 18374 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:24.943838 25234 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:25.026103 25025 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:24.980506 24994 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:25.042373 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.043970 25218 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.044701 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:24.978246 25185 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 1.325s	user 0.006s	sys 0.000s
I20260709 12:19:25.058688 24994 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:25.088112 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2879) took 1668 ms. Trace:
I20260709 12:19:25.088851 24994 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.088461 18448 rpcz_store.cc:276] 0709 12:19:23.419726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.421253 (+  1527us) service_pool.cc:224] Handling call
0709 12:19:23.452047 (+ 30794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.474699 (+ 22652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.474728 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.529037 (+ 54309us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.653208 (+124171us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:23.653275 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.653383 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.977827 (+1324444us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b24001e9cc8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65c159c4 00007f3c65c13acb 00007f3c65dcedc9 00007f3c65dcea55 00007f3c65dce09d 00007f3c68a0010e 00007f3c68a1788c 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221
0709 12:19:25.087816 (+109989us) raft_consensus.cc:1685] finished
0709 12:19:25.087845 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.088055 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6763,"raft.queue_time_us":121070,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":39661,"spinlock_wait_cycles":2777472,"thread_start_us":123707,"threads_started":1,"wal-append.queue_time_us":124084}
I20260709 12:19:25.090263 24994 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.094174 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.094807 18447 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:25.085695 25149 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.095768 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.096803 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.097476 18447 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.096410 25042 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.098076 19038 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
W20260709 12:19:25.100219 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 140084us
I20260709 12:19:25.102996 18448 raft_consensus.cc:1240] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.103046 25042 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.112540 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:24.889572 25173 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:25.028657 25150 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.000s
I20260709 12:19:25.114634 25150 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.002764 25117 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.994086 18370 rpcz_store.cc:276] 0709 12:19:23.551738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.552677 (+   939us) service_pool.cc:224] Handling call
0709 12:19:23.572176 (+ 19499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.784408 (+212232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.784460 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.799281 (+ 14821us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.226748 (+427467us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:24.226807 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.993007 (+766200us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54003f1454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:24.993076 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.993138 (+    62us) raft_consensus.cc:1685] finished
0709 12:19:24.993159 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.993450 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7121,"spinlock_wait_cycles":482280832,"thread_start_us":361395,"threads_started":1,"wal-append.queue_time_us":424573}
I20260709 12:19:25.062181 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.064049 25161 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.064275 25141 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:25.066072 19036 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:25.066810 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:25.039055 25131 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 12:19:25.068329 18296 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.068678 25229 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:25.069243 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.072563 25143 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.073689 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:25.073701 25237 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:19:25.078358 25190 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:24.956614 25221 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.084735 25193 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:25.085170 25233 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:19:25.085628 25226 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.969031 25236 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.128s
I20260709 12:19:25.113624 18448 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:25.124416 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.196969 19038 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:25.126574 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.127812 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.052628 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.135330 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.135123 19007 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:25.137460 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.140437 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.142004 18300 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.176982 19036 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:25.053021 24878 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.202090 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.180956 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.060168 25200 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.178525 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.189700 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.190660 25221 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.192046 18375 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:25.204972 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69924.2us
I20260709 12:19:25.126511 25025 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:25.129050 25141 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.132241 25229 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:25.209095 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.215073 18373 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:25.231431 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.213454 19035 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:25.218842 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.229486 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:25.229722 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 102037us
I20260709 12:19:25.234735 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.236912 18294 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:19:25.239342 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.228971 24878 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.240945 24878 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.241250 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.241547 24878 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.242839 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.244067 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.244913 18443 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.247972 18294 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:25.250624 24878 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.136382 25131 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.255340 24878 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.257231 25131 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.094641 25177 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.260589 25177 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.262343 18366 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:25.263974 18366 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:19:25.264573 18299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42910 (request call id 2453) took 1601 ms (client timeout 1869 ms). Trace:
W20260709 12:19:25.264950 18299 rpcz_store.cc:269] 0709 12:19:23.685586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.685927 (+   341us) service_pool.cc:224] Handling call
0709 12:19:25.264413 (+1578486us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54000a9bd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:25.264505 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151320960}
I20260709 12:19:25.267748 25130 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:25.268959 25130 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:25.269313 25130 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.241427 25245 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:19:25.271490 25245 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:25.271907 25245 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:25.274353 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2892) took 1324 ms. Trace:
I20260709 12:19:25.235857 25042 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:19:25.274829 18445 rpcz_store.cc:276] 0709 12:19:23.949914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.120747 (+170833us) service_pool.cc:224] Handling call
0709 12:19:24.121123 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.489561 (+368438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.274124 (+784563us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54003552d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:25.274291 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831082752}
I20260709 12:19:25.276496 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.276949 24933 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:25.277281 25150 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.291447 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.211244 25247 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:25.210762 25166 raft_consensus.cc:2804] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:25.278795 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.300416 18448 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.282029 18372 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:25.240923 25243 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:25.280414 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:25.296115 18291 raft_consensus.cc:1275] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:19:25.282574 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 47937us
I20260709 12:19:25.328125 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:25.128934 25161 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.337858 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.340477 18339 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.352368 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.355741 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.298079 25143 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.331900 25221 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.363811 18293 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:25.350277 25226 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.272953 25245 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.358429 25150 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.228088 25200 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.389894 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.369268 25134 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.369627 25222 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.370980 25239 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:25.391498 25173 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.371765 25226 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:25.332837 25247 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:25.405083 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.343708 25229 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.362605 25243 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.410637 18375 raft_consensus.cc:2468] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:25.395454 25134 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:25.403067 25226 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.402730 24983 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.362196 25130 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.422881 25134 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.413085 25245 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:25.442787 25056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.005s	sys 0.019s
I20260709 12:19:25.442971 18292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
W20260709 12:19:25.448333 25249 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:19:25.450511 18374 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56140 (request call id 1578) took 1013 ms. Trace:
I20260709 12:19:25.450876 18374 rpcz_store.cc:276] 0709 12:19:24.437595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.006849 (+569254us) service_pool.cc:224] Handling call
0709 12:19:25.007163 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.007978 (+   815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.008007 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.008510 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.205533 (+197023us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:25.205589 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.205678 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.450280 (+244602us) raft_consensus.cc:1685] finished
0709 12:19:25.450302 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.450451 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":196622,"threads_started":1,"wal-append.queue_time_us":333934}
I20260709 12:19:25.448371 18292 raft_consensus.cc:606] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.442270 24983 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.455061 18292 raft_consensus.cc:614] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:25.460980 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 180666us
W20260709 12:19:25.462051 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 55561.5us
I20260709 12:19:25.464569 25173 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.465183 25173 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:25.457368 25056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.005s	sys 0.020s
I20260709 12:19:25.468564 25257 consensus_queue.cc:1059] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:25.472699 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.468989 25173 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.473438 18300 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:25.473912 25162 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:25.474409 25162 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:25.475086 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2883) took 1064 ms. Trace:
I20260709 12:19:25.476272 25117 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:25.476631 25117 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.475740 25162 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.479494 25162 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:25.480815 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.482535 25247 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.445407 25177 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.484488 25177 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.486239 25149 consensus_queue.cc:1059] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:25.488881 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.489494 19035 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:25.491752 25130 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.492367 25173 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.492884 25173 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.494912 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.495507 25194 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.495990 25194 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.497025 25173 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.497558 25243 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.498103 25194 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.498740 18371 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:25.500100 18371 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46418 (request call id 2538) took 1182 ms. Trace:
I20260709 12:19:25.500003 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:25.500607 18371 rpcz_store.cc:276] 0709 12:19:24.318176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.320222 (+  2046us) service_pool.cc:224] Handling call
0709 12:19:25.499947 (+1179725us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000c8fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:25.500052 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2698565376}
I20260709 12:19:25.502490 25200 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.503731 25200 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.504079 25155 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.504187 25200 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.504813 25200 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.509130 25141 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.510334 19036 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.512051 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.487142 25177 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.515983 25161 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.519044 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.519721 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.520515 25117 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.521785 25117 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.522354 25200 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.525930 25177 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:25.526594 25177 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.527132 25177 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.537097 25257 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.537743 25257 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.541077 25257 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.542012 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.471306 25234 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.544663 25205 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.545250 25205 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.547392 25234 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.549057 25234 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:25.539011 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.551589 25206 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:25.552035 25206 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.552280 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.551688 25177 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.553023 25159 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.552904 25206 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.553735 25159 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.554109 25206 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.554454 19009 leader_election.cc:400] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23.
I20260709 12:19:25.554737 19009 leader_election.cc:403] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:19:25.555467 24796 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:25.556896 24796 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:25.557068 25057 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:25.557498 25057 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:25.557673 25125 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:25.557845 24796 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:25.558322 25057 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.555465 25257 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.563751 18448 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:25.564705 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.565346 25019 raft_consensus.cc:3055] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:25.565919 25019 raft_consensus.cc:740] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:25.566342 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.551609 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.570382 18373 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:25.571563 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.572877 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.471171 18447 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.575021 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.575579 25217 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.567088 25019 consensus_queue.cc:260] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.577651 25217 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.579800 25019 raft_consensus.cc:3060] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:25.580838 25217 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.581665 18367 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:25.543677 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.586515 25200 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.587150 25187 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:25.588172 25187 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:25.588505 25187 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:25.588896 25159 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.589416 25159 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.558852 24796 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.592976 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.589282 25187 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.596107 19040 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.599502 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.593637 18264 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.605312 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.605923 19042 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:25.475471 18291 rpcz_store.cc:276] 0709 12:19:24.410910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.417399 (+  6489us) service_pool.cc:224] Handling call
0709 12:19:25.280861 (+863462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.334846 (+ 53985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.474896 (+140050us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54000a9bd0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:25.475031 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984900992}
I20260709 12:19:25.608453 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.615754 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.620618 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.625794 18367 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:25.625964 18368 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:25.626211 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.626924 18445 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 21. Current term is 23. Ops: []
I20260709 12:19:25.627918 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.632454 25234 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:25.639240 25112 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.639636 25187 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:25.642206 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.621613 25057 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.630627 25172 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:19:25.647815 25204 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.649247 25130 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.649403 18373 raft_consensus.cc:1240] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.652880 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2897) took 1488 ms. Trace:
I20260709 12:19:25.660777 18291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:25.682178 18291 raft_consensus.cc:606] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:25.606630 25177 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.662936 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.663800 25159 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.666121 24983 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.512320 25141 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.668677 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.700536 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.701123 18372 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.701795 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.670483 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.703253 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.704169 18366 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 21: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:25.704281 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.705078 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.705478 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.705816 18366 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 22.
I20260709 12:19:25.706692 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.707456 18371 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 23: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:25.708508 19009 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.709857 25141 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.710248 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.677753 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.711038 25141 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:25.711993 25162 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:25.677906 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:25.678838 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.679525 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.714057 18375 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:25.715852 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.627200 25243 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:25.681226 18444 rpcz_store.cc:276] 0709 12:19:24.164434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.372098 (+207664us) service_pool.cc:224] Handling call
0709 12:19:24.372511 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.507801 (+135290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.652557 (+1144756us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54001a5254. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:25.652800 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309551872}
I20260709 12:19:25.627430 25194 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.685792 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.686563 25205 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.687289 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.694984 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.736444 19038 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.736917 18444 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:25.697400 18291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:25.643067 25134 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.704406 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.717629 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.673233 19040 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:25.724124 19009 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.674113 25149 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:25.731918 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.624105 25193 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:25.740175 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.740824 18375 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.741703 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.719430 25204 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.742825 25194 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:25.732331 25130 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:25.763162 25226 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.738482 25243 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.740139 18291 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.763329 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.764540 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:25.750432 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.776453 19040 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.761937 18372 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:25.769086 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.777226 18300 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:25.770704 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.774119 19043 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.765386 18293 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.776491 19038 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:25.761282 24822 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.761755 24878 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.782202 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.787174 18292 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.783331 18295 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.786487 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.789346 18441 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.791245 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.795073 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.773267 25166 raft_consensus.cc:697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:25.774731 25257 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23.
I20260709 12:19:25.802093 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.775658 25194 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:25.799294 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.808521 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:25.811146 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.813988 19040 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:25.826560 18291 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.763993 25149 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.774545 25247 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:19:25.842650 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.844594 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.848034 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:25.850849 18295 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:25.852324 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.854755 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:25.859530 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:25.861804 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:25.862879 18263 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.869642 18291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:25.870214 18291 raft_consensus.cc:606] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:25.870507 18291 raft_consensus.cc:614] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:25.794662 24878 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.858986 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.877391 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:25.880686 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 210673us
I20260709 12:19:25.884656 19042 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 23.
I20260709 12:19:25.844035 25130 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:25.893230 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.887253 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:25.854431 25202 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:25.900696 18298 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:25.902536 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:25.902220 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:25.903877 18338 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:25.869205 25161 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.909471 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:25.908236 18449 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:25.907514 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:25.910878 18374 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.912873 25193 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.913311 25193 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:25.913730 25193 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.914350 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:25.916396 18442 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.847550 25190 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:19:25.908916 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.917897 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:25.921530 18295 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.895418 24933 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:25.886875 25215 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:25.922559 18296 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:25.928551 18442 raft_consensus.cc:1240] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:25.886221 25201 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:25.868845 24996 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:19:25.934751 25042 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:25.940002 25212 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.940850 25190 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:25.942893 25262 log.cc:927] Time spent T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:19:25.919385 25200 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.900638 24987 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:25.947412 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.947497 25194 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:25.949139 25134 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:25.950269 25204 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:25.905830 25172 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:25.934214 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:25.954224 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 252851us
I20260709 12:19:25.959100 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 2884) took 1541 ms. Trace:
I20260709 12:19:26.091325 18415 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.091542 18297 rpcz_store.cc:276] 0709 12:19:24.418049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.418157 (+   108us) service_pool.cc:224] Handling call
0709 12:19:25.958854 (+1540697us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400090454. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:25.959028 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653310208}
I20260709 12:19:25.959865 18291 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:25.953828 24933 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.988492 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:25.910163 24983 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:26.098626 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 224897us
I20260709 12:19:25.953161 18292 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 23.
I20260709 12:19:25.966863 25201 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.958377 25019 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:19:25.990768 25190 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:25.873360 25243 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:25.895008 25166 consensus_queue.cc:237] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.003242 25042 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:26.090251 25169 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.093346 25172 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.094175 25200 raft_consensus.cc:2804] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:26.094651 24796 raft_consensus.cc:2764] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
W20260709 12:19:26.095021 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 160905us
I20260709 12:19:26.116132 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:26.116495 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:26.113814 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 116250us
I20260709 12:19:26.117143 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:25.991804 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:25.995294 25212 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:26.106451 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 198542us
I20260709 12:19:26.118230 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.118822 18440 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:26.123625 18292 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:26.121641 18298 raft_consensus.cc:2393] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 23: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:26.123811 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.105098 25201 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:26.104642 24878 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.135509 25204 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:26.135288 25190 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.136617 25190 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.140544 24933 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.110968 25150 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:19:26.145221 25057 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:26.145745 25057 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.146071 18445 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:26.147961 25057 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.149226 25042 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.149976 25200 raft_consensus.cc:697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:26.151409 25200 consensus_queue.cc:237] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.120564 25239 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:26.174824 24822 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:19:26.118805 25169 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.187400 25149 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.159345 25112 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.189361 25141 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:19:26.192791 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58055.3us
I20260709 12:19:26.201263 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:26.206636 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.214414 24929 raft_consensus.cc:993] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:26.215682 25172 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.209335 25112 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:26.221331 25155 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.221408 25112 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.221984 25155 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:26.222556 25212 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.224025 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.224660 18292 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:26.227823 25112 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.228701 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.197942 24987 consensus_queue.cc:1059] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:19:26.231019 24987 raft_consensus.cc:3055] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.191423 25270 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:26.231487 24987 raft_consensus.cc:740] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:26.231925 25270 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.232617 24987 consensus_queue.cc:260] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.233870 24987 raft_consensus.cc:3060] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.234110 25270 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.237775 18449 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:26.196837 25141 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:26.239436 25141 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.205049 25202 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:26.241019 25202 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.239858 25193 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:26.242934 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.243428 18441 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.243479 25193 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.247865 25177 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:26.247747 25201 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.249832 25226 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.214989 24929 raft_consensus.cc:1081] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:26.263350 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.223039 25155 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.255003 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:26.267910 25187 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.003s
W20260709 12:19:26.258739 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.258630 18296 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:26.191021 18374 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 23.
I20260709 12:19:26.266058 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:26.266464 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.295712 18297 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:26.266867 18292 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:26.255692 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:26.256358 25177 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:26.271097 25029 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:26.271333 25112 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:26.272154 24929 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:26.277729 18195 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 11 to 15, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:26.279835 25169 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:26.282225 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 297) took 15420 ms (client timeout 15000 ms). Trace:
I20260709 12:19:26.284749 25201 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.287863 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:26.312566 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 182744us
I20260709 12:19:26.208444 25057 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:26.320111 19036 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:26.331691 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:26.333142 19042 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:26.292142 25131 raft_consensus.cc:993] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:26.334692 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:26.334270 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.336292 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.345978 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.347683 18264 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:26.344281 25134 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:26.327984 25029 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.349386 25160 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:26.342245 25169 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:26.351035 25131 raft_consensus.cc:1081] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:26.367070 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.344450 18195 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 13, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 13 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:26.351066 25169 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.383617 18445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 3039) took 1175 ms. Trace:
W20260709 12:19:26.385157 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.388283 18445 rpcz_store.cc:276] 0709 12:19:25.207810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.627794 (+419984us) service_pool.cc:224] Handling call
0709 12:19:25.628045 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.146459 (+518414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.380821 (+234362us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54001a6654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:26.383526 (+  2705us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190514432}
W20260709 12:19:26.385773 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 228741us
I20260709 12:19:26.389943 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.353567 25162 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:26.379117 18195 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:26.332597 24929 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.365955 25275 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.398296 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.346940 25057 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:26.373023 25187 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.330901 25151 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:26.414311 25229 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:26.410802 25057 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.418630 25155 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:26.328770 18196 rpcz_store.cc:269] 0709 12:19:10.862124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.862570 (+   446us) service_pool.cc:224] Handling call
0709 12:19:11.284634 (+422064us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:26.282022 (+14997388us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:18.892793 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:18.893552 (+   759us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:18.893598 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:19:18.893665 (+    67us) tablet.cc:662] Decoding operations
0709 12:19:18.915598 (+ 21933us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:18.915677 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:18.915705 (+    28us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:19:18.919210 (+  3505us) tablet.cc:701] Row locks acquired
0709 12:19:18.919256 (+    46us) write_op.cc:260] PREPARE: finished
0709 12:19:18.919403 (+   147us) write_op.cc:270] Start()
0709 12:19:18.919572 (+   169us) write_op.cc:276] Timestamp: P: 1783599558919353 usec, L: 0
0709 12:19:18.919594 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:19:18.931439 (+ 11845us) log.cc:844] Serialized 9857 byte log entry
0709 12:19:19.425255 (+493816us) op_driver.cc:464] REPLICATION: finished
0709 12:19:19.848592 (+423337us) write_op.cc:301] APPLY: starting
0709 12:19:19.855393 (+  6801us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:21.345567 (+1490174us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:21.345774 (+   207us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:21.746638 (+400864us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:21.747180 (+   542us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:19:21.763278 (+ 16098us) write_op.cc:312] APPLY: finished
0709 12:19:21.764076 (+   798us) log.cc:844] Serialized 220 byte log entry
0709 12:19:21.858009 (+ 93933us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:21.961859 (+103850us) write_op.cc:454] Released schema lock
0709 12:19:21.961950 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:21.962586 (+   636us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:21.962621 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":613758,"reported_tablets":46,"spinlock_wait_cycles":571106048,"child_traces":[["op",{"apply.queue_time_us":392079,"apply.run_cpu_time_us":61199,"apply.run_wall_time_us":2141800,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108527,"num_ops":25,"prepare.queue_time_us":293252,"prepare.run_cpu_time_us":19259,"prepare.run_wall_time_us":188808,"raft.queue_time_us":177034,"raft.run_cpu_time_us":87890,"raft.run_wall_time_us":368556,"replication_time_us":505446,"thread_start_us":1019950,"threads_started":5,"wal-append.queue_time_us":364260,"wal-append.run_cpu_time_us":39807,"wal-append.run_wall_time_us":2238204}]]}
I20260709 12:19:26.352907 25217 consensus_queue.cc:1059] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:19:26.426986 25149 raft_consensus.cc:3055] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:26.427457 25149 raft_consensus.cc:740] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:26.428365 25149 consensus_queue.cc:260] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.429863 25149 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:26.398624 25145 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.436599 25145 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.393801 25243 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:19:26.439670 25029 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.439926 25243 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.440409 25243 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:26.441304 25243 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.375931 25186 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:26.442584 25243 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.406778 18447 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:26.393455 25169 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.403239 18440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:26.443926 18440 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:26.444293 18440 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.396399 18195 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:26.466312 18195 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 12 to 13, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:26.470201 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 353091us
I20260709 12:19:26.395339 25162 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:26.476231 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:26.402925 25150 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:19:26.482906 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.501068 25222 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.506167 18443 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.509644 18338 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.484781 25132 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.514825 25132 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.515533 25132 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.533849 25162 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.535305 25239 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.535902 25239 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:26.536015 24929 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.536938 25239 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.538172 25239 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.524755 25282 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:26.546679 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.549624 18374 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:33256
I20260709 12:19:26.550261 18374 raft_consensus.cc:606] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:26.550599 18374 raft_consensus.cc:614] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:26.552716 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.561390 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.562608 18374 raft_consensus.cc:2410] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:26.565022 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.569289 18369 raft_consensus.cc:2393] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 20: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:19:26.568933 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 170515us
I20260709 12:19:26.572708 25177 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.575616 25145 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.570950 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:26.580864 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:26.528947 25202 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.535399 25162 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:26.571900 18195 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 10 to 11, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 11 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:26.588369 24822 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:19:26.592849 18449 raft_consensus.cc:1240] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:26.598104 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.603909 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.605813 18338 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.609203 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.610661 18369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:19:26.614886 18369 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:26.615263 18369 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:26.612601 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.613579 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.624451 18446 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:26.626479 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:26.635807 25284 log.cc:927] Time spent T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:19:26.636452 25278 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:26.637051 25278 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.638024 25132 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.626416 25217 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:26.640426 25217 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:26.640796 25217 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:26.642010 25287 log.cc:927] Time spent T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:19:26.642114 25278 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.645632 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.624756 25194 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:26.647548 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.648185 18372 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:26.649700 24796 consensus_queue.cc:1048] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:26.651938 25215 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.652524 25215 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.655597 24796 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:26.657923 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.659077 18448 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:26.667316 18372 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:26.647902 25194 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.669626 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:26.660769 18440 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.614104 25282 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:26.667541 25290 log.cc:927] Time spent T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:19:26.671360 25194 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.672291 25280 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.672356 25202 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.674593 18338 leader_election.cc:400] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:26.675185 25215 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.665258 18195 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 12 to 17, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 17 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:26.677428 18339 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.677745 25209 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:26.679876 25187 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.614378 25150 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.687791 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.688246 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:26.688623 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 281265us
I20260709 12:19:26.688982 18374 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:26.690074 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.682528 18440 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:26.683176 18195 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 16 to 17, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:26.690083 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.685158 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:26.694891 18195 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:26.686437 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.697531 18447 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:26.687150 18369 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.695734 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:26.705924 25291 log.cc:927] Time spent T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:26.707337 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.711266 18195 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:26.714934 18369 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.717705 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:26.719209 18338 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.723191 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:26.724149 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:26.726105 25275 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.726534 25275 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.726920 25275 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.727262 24983 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:26.727684 24983 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.728057 24983 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:26.690889 25187 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.734915 18440 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:26.734731 25169 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.738070 25155 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:26.716334 25280 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.739049 25280 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:26.739468 25280 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.741537 25150 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:26.742141 25149 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:26.742663 25150 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.742861 25019 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.743341 25019 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:26.743291 25150 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:26.744438 25019 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.745865 25019 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.745925 25150 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.747284 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:26.747968 18440 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:26.690845 25282 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:26.751511 18368 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:26.742695 25149 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:26.753162 18371 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:19:26.753551 18371 pending_rounds.cc:85] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 12
I20260709 12:19:26.753804 18371 pending_rounds.cc:107] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:19:26.753283 25149 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.754022 18371 raft_consensus.cc:2889] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:26.754351 18371 pending_rounds.cc:107] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:19:26.754571 18371 raft_consensus.cc:2889] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:26.761209 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.766404 19042 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.773165 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.773629 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.773826 18296 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:26.775023 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.775907 18368 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:26.779634 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.779881 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.780135 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:26.783550 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:26.784346 18297 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 14: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:26.802284 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:26.803170 19036 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:26.805284 25212 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.805835 25212 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:26.806368 25293 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:19:26.808262 25212 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:26.809317 25279 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:19:26.814529 18448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 2926) took 1307 ms (client timeout 1540 ms). Trace:
I20260709 12:19:26.798861 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:26.815076 18448 rpcz_store.cc:269] 0709 12:19:25.507590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.564486 (+ 56896us) service_pool.cc:224] Handling call
0709 12:19:26.814312 (+1249826us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001860d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:26.814455 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2515315072}
I20260709 12:19:26.797698 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:26.821426 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 72191.3us
W20260709 12:19:26.821648 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:26.823016 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51235.6us
I20260709 12:19:26.830055 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:26.789016 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.779906 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:26.813211 25209 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:26.785480 25194 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.801594 25162 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.822383 25283 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:26.849898 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.862478 18374 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:26.843773 18371 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:26.850780 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:26.863509 18370 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.797088 25159 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:26.842131 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53181us
I20260709 12:19:26.784789 25029 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:19:26.874358 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.871889 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:26.875053 18375 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:26.875602 18371 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.866915 18415 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.873081 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:26.868017 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:26.813901 25243 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:19:26.880003 18415 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.875002 18374 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:26.842033 25169 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.851989 25145 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:26.886126 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.891865 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.855302 25202 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:26.896556 18368 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:26.898885 18415 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:26.900306 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.906728 18292 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:19:26.908540 18366 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:26.867780 25159 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:26.884398 25194 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:26.884845 25229 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.951038 25229 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.953675 25229 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:26.856426 25125 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:19:26.891038 25057 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.958915 18448 raft_consensus.cc:1275] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:26.963016 24822 consensus_queue.cc:1048] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:26.902737 25042 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.919893 18375 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:26.906050 25236 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:26.921941 18371 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:26.893739 25145 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:26.896548 25279 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:26.936050 24983 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:26.937784 25275 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.944162 19036 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:26.947542 25202 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:26.948395 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48222.5us
I20260709 12:19:26.877143 25283 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:26.986313 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:26.915556 25159 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:26.949245 18370 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:26.978619 25145 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:26.969769 25236 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:26.942857 25280 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.013103 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:27.013759 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.015712 18371 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:19:27.020520 25202 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:27.021150 25202 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:27.021727 25295 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:19:27.022279 25202 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.024293 25177 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.006935 25229 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:27.021159 24983 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.027778 25229 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:27.008837 25194 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:19:27.006909 25173 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:27.026258 25275 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.007645 25236 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.037312 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.038086 18375 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
W20260709 12:19:27.038391 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 320725us
I20260709 12:19:27.014791 18338 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:26.995677 25279 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:19:27.046344 18263 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.049710 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:27.028879 25229 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.056763 25132 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:27.058320 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.049396 25217 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.053094 25042 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:19:27.060207 18372 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:27.061901 25217 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.058984 25294 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:27.064353 25294 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:27.065022 25187 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.074887 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.078500 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.079571 19043 raft_consensus.cc:1240] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:27.078526 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.081183 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.081921 19040 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:27.082005 25236 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.061630 25258 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.061599 25169 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:27.084316 25169 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.084736 25169 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.083401 25258 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.071458 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.088677 19037 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:27.095978 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.096210 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.052961 25155 raft_consensus.cc:2804] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:27.066519 24803 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:27.103569 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.105224 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.106029 18373 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:27.104683 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.112145 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.113032 18373 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:27.114235 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:27.117377 19038 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:27.116281 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:27.125102 19007 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.133148 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:27.065301 25294 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.136509 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.139582 18445 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:27.108963 25155 raft_consensus.cc:697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:27.138527 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.110937 24803 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:27.117036 25194 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:27.152587 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.161887 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.128583 25212 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.165205 18293 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:27.168372 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:27.177793 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.178472 18367 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.137483 25132 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.182406 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.159448 25278 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:27.163081 25190 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:27.197297 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.176667 25294 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.171577 25194 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.202451 18296 raft_consensus.cc:1240] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:19:27.178757 24929 raft_consensus.cc:2764] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:19:27.240557 18292 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:27.197710 25190 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:27.203697 25283 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.207739 25194 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:27.237372 25215 consensus_queue.cc:1059] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:19:27.233465 25278 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:27.191792 25132 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:27.261041 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92738.6us
W20260709 12:19:27.265271 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.253356 25190 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.256036 25280 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.293596 19006 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:27.260010 25278 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.257596 25279 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:27.238204 25306 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:19:27.270651 25162 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:27.292387 25239 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:27.311837 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.314306 18370 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.268965 24929 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:19:27.320194 19037 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:27.297134 18367 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:27.262635 25155 consensus_queue.cc:237] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.308104 25186 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:27.326520 25186 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:27.327412 25186 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.331404 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:27.333331 18367 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.335587 18370 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:27.344846 25217 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:27.345422 25217 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.347721 25217 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:27.353217 25305 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:27.257582 24803 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.352998 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.355247 18448 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 22: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:27.366739 25310 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:19:27.375926 25141 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:27.379933 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:27.302690 25279 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.302342 25278 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.387008 25279 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:27.395563 25320 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:27.397205 25132 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.316982 25159 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.401285 25159 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.403494 19042 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:27.408088 25304 log.cc:927] Time spent T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.233s	user 0.003s	sys 0.000s
I20260709 12:19:27.411989 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.413096 19043 raft_consensus.cc:2393] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:27.412174 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.414796 18298 raft_consensus.cc:3055] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:27.415202 18298 raft_consensus.cc:740] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:27.415966 18298 consensus_queue.cc:260] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.417255 18298 raft_consensus.cc:3060] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:27.419929 25258 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.426532 25145 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.429126 25057 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.429548 25194 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.321630 25239 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:27.441243 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 117127us
I20260709 12:19:27.437922 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:27.444746 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.446807 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.465216 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.453603 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.451785 18298 raft_consensus.cc:2468] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:27.464676 19037 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:27.467233 19039 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:27.466120 19043 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:27.465269 19040 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.443602 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:27.473145 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.474814 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.449219 25151 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.473097 18266 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:27.473443 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:27.482297 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:27.478681 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:27.487130 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.487757 18445 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:27.489149 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:27.490118 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.490408 19040 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:27.452411 25187 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:27.493588 18449 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:27.502357 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.503393 18298 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 23: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:27.462502 24822 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:27.502197 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.525774 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:27.449702 25190 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.531527 18448 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.494040 25151 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.483510 24933 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.540968 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.493789 25280 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:27.480247 24796 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:27.539162 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.539558 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.504469 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.546758 18292 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.540241 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.547510 18293 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:27.494971 24929 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:27.484098 25271 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.557866 25271 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.559489 25173 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:27.560099 25173 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:27.534447 25202 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:19:27.561102 25145 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.483845 25283 consensus_queue.cc:1059] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:19:27.561008 25173 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.563934 18440 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.543818 18448 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:27.565414 25239 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.566443 25169 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.568666 25169 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.527671 25296 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:19:27.570250 25296 raft_consensus.cc:3055] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:27.570585 25296 raft_consensus.cc:740] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:27.483599 25132 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:27.493221 25177 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:27.502419 25306 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:27.550432 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 46063.1us
I20260709 12:19:27.550837 18297 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:27.556360 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.588059 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.530680 25187 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.496948 25303 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:27.478606 25019 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:19:27.571419 25296 consensus_queue.cc:260] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.499287 25131 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:19:27.604177 25187 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.598165 25303 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.606199 25278 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.606945 25303 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.576007 25132 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:27.608409 25132 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.609515 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.610153 19044 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:27.538312 24822 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.612421 25194 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.593866 25141 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:19:27.613863 25141 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:27.614319 25141 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:27.586545 25306 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.615679 25306 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.621071 25190 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.542104 25275 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.572544 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:27.631853 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 121030us
I20260709 12:19:27.576356 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.579221 18292 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:27.579957 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.584589 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:27.537524 25308 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:27.560108 25271 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.590947 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:27.590298 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.549779 24933 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.546000 25280 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:27.597626 25151 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.546986 24796 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.544225 24803 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:27.600613 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.648478 19039 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.603020 25019 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:27.648970 18294 raft_consensus.cc:1240] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:27.624711 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.626686 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.594120 25177 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:27.652295 18182 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:27.641247 25275 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.642642 24822 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.645038 18442 raft_consensus.cc:2368] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:27.646135 18264 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.647462 18372 raft_consensus.cc:2393] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 21: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:27.650210 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.648378 24796 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.656749 25308 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:27.661563 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:27.668260 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 77403us
I20260709 12:19:27.669413 18375 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:27.637252 25186 consensus_queue.cc:1059] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:19:27.678120 18264 leader_election.cc:400] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:27.679450 25215 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:27.661530 24803 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.701714 25215 consensus_queue.cc:1243] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:27.669159 25212 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.683482 25334 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.702776 25334 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.703994 18295 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:27.704397 18295 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:27.704097 25308 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.704936 25334 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:27.674641 25151 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:27.649348 25280 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.651295 25019 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:27.685925 18443 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:27.687175 25177 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.695125 24933 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.697736 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.711400 18375 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:27.699561 18369 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.650718 25141 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.713179 25236 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.713558 25141 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.705163 18295 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.716042 18295 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.716850 25275 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.717132 25239 raft_consensus.cc:3055] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:27.717540 25239 raft_consensus.cc:740] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:27.718324 25239 consensus_queue.cc:260] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.704394 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.719326 18300 raft_consensus.cc:1275] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:27.719643 18373 raft_consensus.cc:2393] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:27.721189 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.721705 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:27.724475 24933 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:27.724896 24933 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.725231 24933 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:27.727838 25201 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:19:27.709630 18374 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:27.730589 25333 log.cc:927] Time spent T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:19:27.730409 25306 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.700949 18264 leader_election.cc:403] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:19:27.731962 25209 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:27.733999 25019 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.735292 25019 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:27.736127 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 58208.8us
I20260709 12:19:27.739488 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.739583 25134 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.742197 25294 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.742659 25294 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.744109 25301 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:19:27.710047 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 48540.3us
I20260709 12:19:27.746136 25322 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:27.746279 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.751770 25303 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.754490 25209 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
W20260709 12:19:27.735302 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 267177us
I20260709 12:19:27.727685 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.765758 18295 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:27.783442 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:27.784700 18338 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:27.673070 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.773391 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.778283 18264 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.782233 19039 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:27.789326 25177 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:27.789824 25177 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.790280 25177 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:27.765923 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.766417 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:27.793104 18442 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.703395 24803 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.786309 18291 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:19:27.796309 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.796097 19006 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.775622 24987 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.798579 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:27.800079 19040 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:27.778283 25239 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.800879 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.801041 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:27.794924 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 121942us
I20260709 12:19:27.807811 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.806324 19043 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
W20260709 12:19:27.795200 25243 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:19:27.808300 18264 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:27.810194 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.785715 25222 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.835762 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.840793 24933 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.842113 18297 raft_consensus.cc:2368] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17.
I20260709 12:19:27.795488 25155 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.846881 18442 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:27.787485 24929 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:27.804368 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:27.804837 25301 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.854537 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:27.854655 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:27.851982 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43749.9us
I20260709 12:19:27.855443 18292 raft_consensus.cc:2368] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:19:27.853451 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:27.854014 18300 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:27.856642 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:27.856899 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.858332 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.818276 25306 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.809629 25283 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:27.863125 25155 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.867125 25125 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:27.866017 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:27.812853 25159 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.868975 24929 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.869567 25308 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:27.862120 25301 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.874558 25301 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.865187 18295 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:27.812633 25209 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.881060 25294 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.778026 25322 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.881481 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.882145 18448 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:27.882608 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.883448 25322 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.883888 25159 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.886109 19040 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.886642 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.891573 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.892469 18369 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:27.894392 19009 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.895928 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:27.891038 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.812389 25149 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:19:27.899407 18442 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:27.911301 25186 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.920847 25186 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.916204 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.923959 18448 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:27.927070 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:27.927338 19038 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:27.927747 18369 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 23.
I20260709 12:19:27.928992 25134 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:27.929064 19009 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:27.929557 25134 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:27.931128 25334 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:27.931594 25334 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.931993 25334 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:27.932623 24933 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.852201 25323 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.930538 25134 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.810626 24987 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:27.937600 19040 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:27.960049 25296 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:27.935554 25186 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.963435 25258 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:19:27.976918 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:27.977664 18443 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 23.
I20260709 12:19:27.960300 25132 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.979910 25132 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:27.981045 25303 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:27.981786 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:27.953629 25162 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:27.982434 18369 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.982921 25162 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.986378 25162 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.987630 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:27.955499 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 151200us
I20260709 12:19:27.984438 25177 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:27.897974 25281 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.960592 25190 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:19:27.988329 18442 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.953392 25294 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:27.993848 18369 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:28.017042 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:28.016829 19040 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:28.018500 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:28.019029 18374 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:28.019858 18369 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:19:27.997944 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:28.000484 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:28.016786 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:28.006063 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 125353us
I20260709 12:19:28.021754 19040 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:28.008968 19038 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:28.010072 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:28.011013 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:28.022819 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:28.023471 18375 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.023578 18369 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.947257 19039 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:28.028091 19006 leader_election.cc:400] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17.
I20260709 12:19:28.028450 19006 leader_election.cc:403] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.028311 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:28.029946 18412 leader_election.cc:400] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
I20260709 12:19:28.030341 18412 leader_election.cc:403] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.033486 18367 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:28.034021 18366 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:28.032133 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:28.032574 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 164090us
I20260709 12:19:28.042330 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:28.044420 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:28.045087 18448 raft_consensus.cc:2368] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21.
I20260709 12:19:28.048424 18300 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:28.054502 18442 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:28.005944 25308 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:28.016443 25294 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.058363 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:28.059695 18264 leader_election.cc:400] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21.
I20260709 12:19:28.068289 18264 leader_election.cc:403] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.067927 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:28.070209 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:28.071324 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:27.977671 24796 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:28.073315 18374 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:28.075839 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:28.079365 18291 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:28.087839 18369 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:28.087749 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:28.096513 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:28.112035 18375 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:28.059993 24822 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:28.070668 25194 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:28.071764 25294 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:28.118450 24822 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:28.125411 25177 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.126180 25347 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:28.128415 25236 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:28.117070 25279 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:19:28.082589 25151 consensus_queue.cc:1048] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:19:28.075613 25301 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.075428 25217 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.063283 25019 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:19:28.091320 24983 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:28.129460 25134 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.065883 25149 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
W20260709 12:19:28.154299 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 66668.7us
I20260709 12:19:28.155889 25177 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17.
W20260709 12:19:28.154282 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 124466us
I20260709 12:19:28.188830 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:28.189977 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:28.129956 25229 consensus_queue.cc:1059] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:28.153780 25222 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:28.070984 25345 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 12:19:28.192108 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:28.191617 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 118662us
I20260709 12:19:28.160121 24933 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:28.082178 25322 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.077s
I20260709 12:19:28.210099 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:28.216365 18375 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:28.163298 25347 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
W20260709 12:19:28.224226 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 156638us
I20260709 12:19:28.203369 25194 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:28.156467 25236 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:28.228699 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:28.231793 18369 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:28.220744 25177 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:28.237010 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 150312us
I20260709 12:19:28.203600 25169 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:19:28.159739 25308 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.234246 25209 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.230479 24987 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.253765 18266 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:28.256129 25357 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:28.305576 25352 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:19:28.307951 25169 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:28.221525 24822 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.253840 25177 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:28.314121 24796 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.314656 25177 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:28.316000 25177 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:28.319072 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 60206us
I20260709 12:19:28.327256 18298 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:19:28.347880 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 94185us
I20260709 12:19:28.352564 25194 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:28.347527 25345 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:28.315091 25162 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:28.458266 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 107159us
I20260709 12:19:28.470481 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:28.446424 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 254384us
I20260709 12:19:28.441383 19043 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:28.367146 25351 log.cc:927] Time spent T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:19:28.477133 25162 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.417582 25347 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.478960 24983 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21.
I20260709 12:19:28.480690 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:28.481623 18442 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:28.445770 25280 consensus_meta.cc:287] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.740s	user 0.005s	sys 0.009s
I20260709 12:19:28.366873 25145 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.455155 25301 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 16. Current term is 17.
I20260709 12:19:28.435932 25363 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.459779 25294 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.505244 19044 raft_consensus.cc:2368] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 22. Current term is 23.
I20260709 12:19:28.506022 25294 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.364326 25194 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:28.488070 25132 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:28.509960 25132 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:28.512043 25132 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.513546 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:28.514189 18444 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:28.515501 25236 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:28.515903 25360 log.cc:927] Time spent T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:28.366449 25345 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.501765 25280 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.456939 25169 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:19:28.521837 25345 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.488051 25356 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:28.522490 25280 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.522902 25356 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.523394 18368 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:28.500560 25145 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.451833 25357 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:28.526371 19037 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:28.528919 18442 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:28.533200 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:28.532632 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 66476.7us
I20260709 12:19:28.525168 25145 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:28.533217 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 59667.1us
I20260709 12:19:28.537451 18444 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 23.
I20260709 12:19:28.545056 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:28.546130 19040 raft_consensus.cc:2393] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:28.537915 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:28.548614 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:28.551921 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:28.552642 19036 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:28.555595 19038 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
W20260709 12:19:28.556245 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 315057us
I20260709 12:19:28.558785 25283 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:28.525238 25356 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.553808 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:28.566576 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:28.567368 18444 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:28.567878 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:28.575472 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:28.576998 18444 raft_consensus.cc:2393] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:28.585443 25162 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:28.586129 25162 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:28.586664 25278 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.587070 25162 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:28.590260 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52458.3us
I20260709 12:19:28.585641 25357 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.593964 18338 leader_election.cc:400] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:28.594399 18338 leader_election.cc:403] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.597225 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:28.598945 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:28.599448 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:28.601877 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:28.602562 19037 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:28.599542 18448 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.606369 18294 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:28.612157 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:28.614071 19040 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:28.615393 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:28.614516 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:28.622915 18291 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:19:28.623607 18441 raft_consensus.cc:2368] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:28.578931 25303 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:28.620342 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:28.629774 18448 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:28.527710 25363 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:28.595803 25348 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:28.636395 25334 consensus_meta.cc:287] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.000s
I20260709 12:19:28.617552 25155 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.616952 25222 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:28.610987 24796 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:19:28.662809 18292 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:19:28.638092 25149 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.757944 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:28.777665 18298 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.641136 25303 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:28.658857 25177 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:28.675050 18291 raft_consensus.cc:1275] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:28.776670 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:28.789685 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 242517us
I20260709 12:19:28.662091 25334 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.803331 19036 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:19:28.852285 19043 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:19:28.805924 25380 log.cc:927] Time spent T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:19:28.653025 25348 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:28.862551 18298 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
W20260709 12:19:28.859470 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 238926us
I20260709 12:19:28.799468 25019 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:19:28.785784 25222 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.811837 24796 consensus_queue.cc:1059] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:19:28.878680 18442 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:28.881915 18298 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:28.880365 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:28.870988 25355 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:28.864071 25358 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.918915 25358 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.893936 19042 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:28.921137 25358 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.895331 18339 leader_election.cc:400] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:28.895788 25339 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:28.897120 25239 raft_consensus.cc:3055] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:28.923257 25339 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:28.923395 25239 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:28.897656 25392 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:19:28.924013 25363 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:28.898691 25390 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:28.924207 25339 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:28.908444 25348 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.913156 18374 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:28.805078 25303 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:28.934203 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:28.934985 18369 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:28.916148 25334 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:28.917080 25222 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:28.890997 25166 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:19:28.937425 18374 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:28.949136 19037 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:28.952522 25390 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:28.955148 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 174870us
I20260709 12:19:28.958530 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:28.959721 19042 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:19:28.955580 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:28.999905 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 144492us
I20260709 12:19:28.999272 25363 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:28.955781 25145 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.000768 25355 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:19:29.009501 25283 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.013815 25363 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.023783 25363 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.043917 25145 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.023336 25283 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.048671 25283 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.021272 25173 raft_consensus.cc:3055] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.050107 25173 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:29.050942 25173 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.053211 19039 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59422 (request call id 1210) took 1195 ms. Trace:
W20260709 12:19:29.054175 25350 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.928s	user 0.006s	sys 0.004s
I20260709 12:19:29.057009 25355 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.057638 25355 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:29.059283 25374 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.288s	user 0.000s	sys 0.004s
I20260709 12:19:29.062680 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.065157 25355 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:29.053776 19039 rpcz_store.cc:276] 0709 12:19:27.857784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.858013 (+   229us) service_pool.cc:224] Handling call
0709 12:19:29.052990 (+1194977us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b54002166d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:29.053129 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155645824}
I20260709 12:19:29.071898 25229 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.072582 25229 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.075145 25229 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.083117 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:29.083761 18441 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:29.092160 18441 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 24.
I20260709 12:19:29.136013 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.137058 19039 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:29.141202 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.142268 19039 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:29.123301 25155 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18.
I20260709 12:19:29.153535 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:29.159788 25155 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.160477 25155 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.164307 25155 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.139030 25283 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.138841 25159 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.178409 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:29.171959 19007 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:29.149252 24796 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.184641 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.186735 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.189504 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:29.191767 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:29.198014 25187 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:29.185410 25390 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.200278 25125 raft_consensus.cc:2804] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:29.200794 25125 raft_consensus.cc:697] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:29.201702 25125 consensus_queue.cc:237] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.181161 25283 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.204725 25365 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:19:29.205273 25283 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.184890 24796 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.211588 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.212596 18441 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:29.212759 24796 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.191691 25177 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:19:29.214700 25187 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:29.215761 25187 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.214321 25236 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:29.214329 25411 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:29.231583 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 224539us
I20260709 12:19:29.232681 18264 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:29.183082 25159 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.236111 25411 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:29.237371 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.238227 18441 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:29.239516 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:29.240047 18444 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.235049 25159 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:29.241647 25239 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.242835 25239 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.243546 25401 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:29.245594 25390 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:29.237068 25411 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.249310 25363 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.251415 25131 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:19:29.251748 19036 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:29.253916 18368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56140 (request call id 1621) took 1375 ms. Trace:
I20260709 12:19:29.254280 18368 rpcz_store.cc:276] 0709 12:19:27.878756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.886384 (+  7628us) service_pool.cc:224] Handling call
0709 12:19:29.253768 (+1367384us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400395654. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:29.253864 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462849408}
I20260709 12:19:29.256356 18440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48170 (request call id 2981) took 1211 ms. Trace:
I20260709 12:19:29.256752 18440 rpcz_store.cc:276] 0709 12:19:28.044866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.044992 (+   126us) service_pool.cc:224] Handling call
0709 12:19:28.068872 (+ 23880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.155921 (+ 87049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.155951 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.155995 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:28.156034 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.256145 (+1100111us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b5400375854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:29.256189 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.256304 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199746304}
I20260709 12:19:29.261145 18449 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:29.270336 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.271907 18296 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:29.273662 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.274410 18299 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 14.
I20260709 12:19:29.275626 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.288065 18444 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
W20260709 12:19:29.294397 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 122514us
I20260709 12:19:29.300236 18298 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.300499 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.301251 18299 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:29.303326 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.306954 18442 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:29.309625 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.266075 25363 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:29.315820 25173 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.318688 25357 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:29.323493 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59430 (request call id 1391) took 1316 ms. Trace:
I20260709 12:19:29.341671 19044 rpcz_store.cc:276] 0709 12:19:28.007094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.007255 (+   161us) service_pool.cc:224] Handling call
0709 12:19:29.323259 (+1316004us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540066d854. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:29.323424 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094742400}
W20260709 12:19:29.383106 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 69440.6us
I20260709 12:19:29.300992 25285 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.430438 25413 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:19:29.438732 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 87027.1us
W20260709 12:19:29.404690 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65695.8us
W20260709 12:19:29.442454 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 57632.8us
W20260709 12:19:29.476682 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 284961us
I20260709 12:19:29.390301 25131 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:19:29.480098 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:29.494065 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:29.494745 18298 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:29.498821 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 54009us
I20260709 12:19:29.507067 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:29.510617 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:29.515018 25364 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:29.517817 25217 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:29.518280 25217 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:29.519217 25217 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.520511 25217 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.521070 25236 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:29.512331 24796 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.523528 24796 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.524797 25364 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:29.525336 25364 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.525785 24796 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.527649 25364 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.508740 25202 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:29.532322 25202 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.534557 25202 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:29.535989 25409 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:19:29.537236 18443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1627) took 1555 ms. Trace:
I20260709 12:19:29.537585 18443 rpcz_store.cc:276] 0709 12:19:27.982319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.991014 (+  8695us) service_pool.cc:224] Handling call
0709 12:19:28.019607 (+ 28593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.019985 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:28.020018 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:28.943481 (+923463us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b2c000c0988. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:19:28.943933 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.172407 (+228474us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:29.172475 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.172569 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.536951 (+364382us) raft_consensus.cc:1685] finished
0709 12:19:29.536973 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.537177 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46101,"raft.queue_time_us":38958,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7903,"spinlock_wait_cycles":3828608,"thread_start_us":228114,"threads_started":1,"wal-append.queue_time_us":354264}
I20260709 12:19:29.509778 25345 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.539638 25345 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.541754 25345 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.544736 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.545413 19037 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:29.479625 25349 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:19:29.551153 25215 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.551759 25215 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:29.552780 25215 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.452582 25285 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.557466 18295 raft_consensus.cc:1240] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:19:29.556604 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.569628 25397 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.581161 18296 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:29.580161 25019 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:29.582127 25285 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:29.581279 25397 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.578116 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.582186 25019 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.584496 25283 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:29.584673 25019 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.578116 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.585249 25019 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:29.585064 25283 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.585954 19038 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:29.586076 25222 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.586619 25222 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.588291 25283 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.588725 25222 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.589010 25283 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.589942 18338 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:29.591261 25155 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.590838 25019 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.597793 18444 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:29.600593 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:29.557394 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.603395 25339 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.601948 18293 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:29.589661 25283 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:29.577953 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 70986.1us
I20260709 12:19:29.611860 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.612632 18446 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:29.604259 25019 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:29.603993 25339 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.535121 25202 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:29.628152 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 81221us
I20260709 12:19:29.633325 18338 leader_election.cc:400] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 22. Current term is 23.
I20260709 12:19:29.633663 18338 leader_election.cc:403] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:29.633956 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.634793 18368 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:29.636965 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.637534 18366 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:29.637887 18369 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:19:29.629081 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 59239us
I20260709 12:19:29.644832 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.645212 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.647941 18298 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
W20260709 12:19:29.607259 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 139336us
I20260709 12:19:29.657150 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.733074 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.733796 18369 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:29.642932 25019 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.612665 25335 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:19:29.738229 19038 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:19:29.738481 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 94773.2us
I20260709 12:19:29.740434 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:29.740872 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:29.751780 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:29.751783 18446 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:29.741858 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.744814 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.748133 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.674338 24796 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:29.755362 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:29.759630 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.762295 18298 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 23.
I20260709 12:19:29.756095 18412 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:29.729663 25151 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.729452 25393 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.779745 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:29.780396 18366 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.781503 25345 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.782004 25345 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.782406 25345 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.745311 25222 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:19:29.787782 18366 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:29.744410 25279 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:29.730409 25354 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:19:29.761055 24796 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.767409 25159 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.776039 25391 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:29.825024 18295 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:29.821981 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:29.832191 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 80318.4us
I20260709 12:19:29.834117 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:29.754016 25019 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:29.837754 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.839191 18366 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:19:29.848541 18340 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.821019 24796 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:29.860615 18369 raft_consensus.cc:2410] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:19:29.854979 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:29.861486 18366 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.869601 18366 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:29.794112 25151 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.872957 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.799170 25393 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:29.857679 25239 consensus_meta.cc:287] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.000s
I20260709 12:19:29.878407 18370 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:29.879595 25155 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:29.885598 25426 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 12:19:29.892283 25421 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:19:29.893998 25423 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:19:29.898445 25345 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.901064 25283 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:29.901863 25283 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:29.902278 25283 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.902690 25283 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.905309 25151 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:29.911405 25424 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.176s	user 0.004s	sys 0.000s
I20260709 12:19:29.851457 25391 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:29.817874 25279 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:29.875628 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:29.946780 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:29.920419 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 123591us
I20260709 12:19:29.949894 18371 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:29.949196 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:29.830469 25159 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:29.959400 18371 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:29.965283 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:29.860085 25236 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:19:29.969630 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:29.971522 25279 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:29.951591 25239 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.973618 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42910 (request call id 2497) took 1376 ms. Trace:
I20260709 12:19:30.003422 18372 raft_consensus.cc:3060] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:30.005827 25159 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.003299 18297 rpcz_store.cc:276] 0709 12:19:28.597163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.597981 (+   818us) service_pool.cc:224] Handling call
0709 12:19:28.598274 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.853834 (+255560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.853874 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.627446 (+773572us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b2c000cd878. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c67775a11 00007f3c67785372 00007f3c68eb7a01 00007f3c68e7c43a 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2
0709 12:19:29.627843 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.816892 (+189049us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:29.817025 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.819508 (+  2483us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.973311 (+153803us) raft_consensus.cc:1685] finished
0709 12:19:29.973335 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.973554 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64898,"raft.queue_time_us":71217,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":78804,"spinlock_wait_cycles":2427264,"thread_start_us":287504,"threads_started":2,"wal-append.queue_time_us":316737}
I20260709 12:19:30.008275 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:29.974761 24796 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.015136 25432 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.003073 25339 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.010423 25239 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.101948 25433 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:30.043960 25432 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:30.175638 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 434816us
I20260709 12:19:30.036085 25391 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.233407 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:30.239156 19044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.005s	sys 0.006s
W20260709 12:19:30.239722 19044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.007s
I20260709 12:19:30.240943 18372 raft_consensus.cc:2468] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 24.
W20260709 12:19:30.241945 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 279096us
I20260709 12:19:30.251855 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.225566 24983 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.224681 25433 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.242122 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.271760 18266 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:30.259298 25428 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.250s	user 0.000s	sys 0.010s
I20260709 12:19:30.287516 19036 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:30.289942 25432 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.290599 25432 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.250384 25357 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:30.295033 18266 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:30.311292 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:30.317324 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.317631 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.318209 18442 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:30.281788 24983 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.325287 25029 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:30.304610 24933 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:30.326853 24933 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:30.327073 25431 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.327366 24933 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.328022 24796 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:30.330150 25222 consensus_meta.cc:287] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.004s
I20260709 12:19:30.330611 25222 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 22. Current term is 23.
I20260709 12:19:30.297619 25432 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.337311 25432 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.337404 25393 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.311259 25401 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:30.341142 25401 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:30.341531 25401 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.342217 24933 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.343437 25222 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:30.344084 25222 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.344481 25222 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:30.343538 25345 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.347711 24933 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.348702 25397 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.349253 18370 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 23.
I20260709 12:19:30.354770 25397 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:30.355350 25397 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.357399 25397 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:30.361371 25217 consensus_meta.cc:287] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.003s
I20260709 12:19:30.365430 25283 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.309976 25202 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:30.257287 25339 consensus_queue.cc:1048] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:19:30.282553 25433 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.304965 25019 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.390873 18368 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:30.327152 24983 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.305150 25357 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.399207 18372 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
W20260709 12:19:30.401028 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.417726 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:30.418169 18442 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.418359 18446 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:30.387992 25422 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.729s	user 0.000s	sys 0.012s
W20260709 12:19:30.388149 25410 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.003s
I20260709 12:19:30.414526 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.414288 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:30.460803 18375 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:19:30.463446 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49269.2us
I20260709 12:19:30.463793 18374 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:19:30.464744 25339 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.403028 25202 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.466909 25019 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:30.472756 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.473630 18442 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:30.468915 25019 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:30.491690 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 239900us
I20260709 12:19:30.467756 25239 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.472944 18446 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
W20260709 12:19:30.495950 25442 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:19:30.501572 25355 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.496506 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.546365 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:30.507838 25447 log.cc:927] Time spent T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:19:30.508213 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.510058 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.510418 18299 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.512490 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.547046 18294 raft_consensus.cc:3060] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:30.549431 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.550544 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.514660 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.518821 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.520012 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.551784 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:30.480352 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:30.521625 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:30.554454 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 74233.2us
I20260709 12:19:30.521382 25389 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:30.535929 25418 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.965s	user 0.004s	sys 0.000s
I20260709 12:19:30.540108 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:30.476348 25173 consensus_meta.cc:287] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Time spent flushing consensus metadata: real 1.094s	user 0.000s	sys 0.022s
I20260709 12:19:30.547894 18338 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.568822 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.554539 18296 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 23.
I20260709 12:19:30.533492 18366 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:19:30.490918 25019 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:30.557255 19042 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59426 (request call id 1641) took 1295 ms. Trace:
I20260709 12:19:30.573423 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.555217 18339 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:30.564486 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:30.469671 25339 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.536502 25155 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.544045 25217 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.623275 25217 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.625703 25217 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.626264 25217 raft_consensus.cc:2804] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:30.626780 25217 raft_consensus.cc:697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:30.628715 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.543582 25187 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:30.629633 18449 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:30.629242 25217 consensus_queue.cc:237] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.629931 25187 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.631323 25239 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:30.631500 25440 log.cc:927] Time spent T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:19:30.632342 25187 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.633404 19044 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1221) took 1149 ms. Trace:
W20260709 12:19:30.633647 25443 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:19:30.609819 25432 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:30.633803 19044 rpcz_store.cc:276] 0709 12:19:29.483702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.484569 (+   867us) service_pool.cc:224] Handling call
0709 12:19:29.485027 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:29.485486 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.485513 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:29.490789 (+  5276us) log.cc:844] Serialized 52 byte log entry
0709 12:19:30.240234 (+749445us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:30.240297 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:30.240406 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:30.630874 (+390468us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:30.633105 (+  2231us) raft_consensus.cc:1685] finished
0709 12:19:30.633136 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.633347 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98890,"raft.queue_time_us":3774,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":13460,"spinlock_wait_cycles":14626176,"thread_start_us":641834,"threads_started":1,"wal-append.queue_time_us":896604}
I20260709 12:19:30.636475 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.489892 25454 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:19:30.562016 25355 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.641393 25141 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:30.651909 25141 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:30.652552 25419 log.cc:927] Time spent T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.075s	user 0.002s	sys 0.000s
I20260709 12:19:30.654266 25141 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:30.656072 25357 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.659865 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.609876 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:30.610590 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 55475.6us
I20260709 12:19:30.569872 25345 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:30.614492 25451 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:30.559520 25431 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:30.614974 25369 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.616036 25339 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.616439 25155 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.618266 25401 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.621634 25229 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.662597 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.699582 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.662992 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.664181 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:30.646191 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.647322 25303 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.702787 18443 raft_consensus.cc:2368] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18.
W20260709 12:19:30.703666 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 57526.1us
I20260709 12:19:30.673213 18294 raft_consensus.cc:2468] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
W20260709 12:19:30.675818 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 66027.9us
I20260709 12:19:30.678023 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.679447 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.681180 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:30.573730 19042 rpcz_store.cc:276] 0709 12:19:29.261665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:29.262470 (+   805us) service_pool.cc:224] Handling call
0709 12:19:29.262993 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:29.263662 (+   669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:29.263725 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.288157 (+ 24432us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.732799 (+444642us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:29.753260 (+ 20461us) log.cc:844] Serialized 14 byte log entry
0709 12:19:29.777050 (+ 23790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.777141 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.906621 (+129480us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b18003f7840. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c6cd5058f 00007f3c65d8f571 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd
0709 12:19:30.513512 (+606891us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18003f7840. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f3f3 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:19:30.557017 (+ 43505us) raft_consensus.cc:1685] finished
0709 12:19:30.557037 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:30.557185 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66761,"prepare.queue_time_us":27224,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":26884,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":9689,"spinlock_wait_cycles":62607232,"thread_start_us":325598,"threads_started":1,"wal-append.queue_time_us":497123}
I20260709 12:19:30.575703 18293 raft_consensus.cc:606] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:30.646570 25355 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.646920 25283 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:30.646332 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 695909us
I20260709 12:19:30.714731 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:30.641757 25202 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:30.573225 25397 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.129s
I20260709 12:19:30.709045 19036 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 23.
I20260709 12:19:30.709071 19038 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.639179 25222 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.718636 25457 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.708645 25229 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.720220 25222 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.721626 25229 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.723680 25401 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.568416 25389 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.726161 25389 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.701582 25369 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.649391 25019 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.678960 25357 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.680157 25431 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.642256 25445 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:19:30.550729 25453 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.710412 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.746322 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:30.728911 18298 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.748675 25162 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:30.748966 24822 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:30.711938 25301 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:19:30.757184 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45702.9us
I20260709 12:19:30.747633 25431 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.701216 24796 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.647958 25354 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:19:30.768977 18448 raft_consensus.cc:1275] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:19:30.769449 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.745078 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:30.766213 18298 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:30.734174 19038 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
W20260709 12:19:30.771870 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 463319us
W20260709 12:19:30.772728 18195 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 298) took 38256 ms (client timeout 15000 ms). Trace:
I20260709 12:19:30.773916 18448 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:19:30.775660 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.712580 25303 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.766395 25132 raft_consensus.cc:993] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:30.781719 18368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:33256
I20260709 12:19:30.719200 25457 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.784427 18368 raft_consensus.cc:606] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:30.784900 18368 raft_consensus.cc:614] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:30.777858 25125 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.786861 25433 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.787127 25125 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.787663 25369 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.789268 25433 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.751441 25159 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:19:30.789755 24822 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:30.789896 25301 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:30.790405 24822 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.789407 25125 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:30.768477 25141 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:30.791823 25141 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.792187 25141 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:30.792317 25019 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.793380 24822 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.766584 25285 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:30.794416 25285 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.790514 25301 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.790854 25159 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.794886 25285 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:30.795145 25159 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.796443 25301 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.794262 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.797441 25159 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.797679 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.798453 18372 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:30.800791 25303 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:30.781533 25162 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:30.801381 25369 raft_consensus.cc:493] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.801788 25162 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.802214 19036 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:30.802937 19036 raft_consensus.cc:606] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:30.803380 19036 raft_consensus.cc:614] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:30.804170 25162 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.806211 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:30.801932 25369 raft_consensus.cc:515] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.807261 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.822036 18372 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:30.751731 25357 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.816380 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.817077 25212 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.817839 25132 raft_consensus.cc:1081] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:30.832984 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.806167 25155 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.818580 25190 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:30.847164 25190 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.819087 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.820511 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:30.849493 19040 raft_consensus.cc:2368] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20.
I20260709 12:19:30.823688 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:30.850049 25190 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.817984 19042 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:30.851594 25166 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:30.779158 18195 rpcz_store.cc:269] 0709 12:18:52.516166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.516371 (+   205us) service_pool.cc:224] Handling call
0709 12:19:30.772638 (+38256267us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:27.321230 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:27.368500 (+ 47270us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:27.368556 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:19:27.368573 (+    17us) tablet.cc:662] Decoding operations
0709 12:19:27.369624 (+  1051us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:27.369724 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:27.369753 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:19:27.370636 (+   883us) tablet.cc:701] Row locks acquired
0709 12:19:27.370687 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:19:27.370847 (+   160us) write_op.cc:270] Start()
0709 12:19:27.370999 (+   152us) write_op.cc:276] Timestamp: P: 1783599567370780 usec, L: 0
0709 12:19:27.371020 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:19:27.503487 (+132467us) log.cc:844] Serialized 4048 byte log entry
0709 12:19:27.660514 (+157027us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c65d720ac 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b
0709 12:19:28.434028 (+773514us) op_driver.cc:464] REPLICATION: finished
0709 12:19:28.610409 (+176381us) write_op.cc:301] APPLY: starting
0709 12:19:28.628203 (+ 17794us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:28.941029 (+312826us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:28.941054 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:29.106431 (+165377us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:29.106774 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:19:29.106859 (+    85us) write_op.cc:312] APPLY: finished
0709 12:19:29.107273 (+   414us) log.cc:844] Serialized 97 byte log entry
0709 12:19:29.107483 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:29.147664 (+ 40181us) write_op.cc:454] Released schema lock
0709 12:19:29.149620 (+  1956us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:29.190666 (+ 41046us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:29.219476 (+ 28810us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":611177,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":130914,"apply.run_cpu_time_us":17835,"apply.run_wall_time_us":589598,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45887,"num_ops":10,"prepare.queue_time_us":282877,"prepare.run_cpu_time_us":34820,"prepare.run_wall_time_us":571084,"raft.queue_time_us":210129,"raft.run_cpu_time_us":7782,"raft.run_wall_time_us":95345,"replication_time_us":1062796,"spinlock_wait_cycles":32011136,"thread_start_us":556081,"threads_started":4,"wal-append.queue_time_us":416919,"wal-append.run_cpu_time_us":31144,"wal-append.run_wall_time_us":2262306}]]}
W20260709 12:19:30.855011 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 109997us
I20260709 12:19:30.845796 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.852618 25369 leader_election.cc:290] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:30.857484 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49557.6us
I20260709 12:19:30.860787 25019 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:30.889587 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.888402 25159 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:30.866307 18298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:30.890635 25159 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.754897 25453 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.880285 25389 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.882210 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:30.886992 18263 leader_election.cc:400] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20.
I20260709 12:19:30.898525 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.899505 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.856323 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:30.901619 18263 leader_election.cc:403] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:19:30.902686 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46445.5us
I20260709 12:19:30.900969 18264 leader_election.cc:400] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18.
I20260709 12:19:30.903407 18264 leader_election.cc:403] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:19:30.908790 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:30.911800 18264 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:30.913810 18444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:30.914052 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:30.914435 18444 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:30.914896 18444 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.915527 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.915606 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:30.916214 19040 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:30.922137 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.925273 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:30.933518 18338 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.916747 25029 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20.
I20260709 12:19:30.944597 18266 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:30.945622 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.874785 18198 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:30.952504 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:30.889402 25019 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.948168 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.955539 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:30.961615 18294 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.955090 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:30.962241 18444 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:30.955693 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:30.964095 18448 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:30.964818 18338 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:30.965020 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:30.965644 19040 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.943609 25125 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:19:30.967082 25454 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:30.943773 25296 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.968351 25296 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.968755 18444 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:30.970366 25296 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.971572 18294 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:30.964627 25029 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:19:30.984113 25357 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.991046 25457 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.845024 25212 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:30.943463 25308 raft_consensus.cc:2764] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:30.993427 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.994220 19041 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:30.994791 25212 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:30.999482 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:30.917212 25456 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:30.943655 25190 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227) with higher term. Message: Invalid argument: T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18.
I20260709 12:19:30.943396 25159 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:30.998972 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:31.008723 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53117.3us
I20260709 12:19:31.019744 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.020385 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.033592 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:30.907275 25301 consensus_queue.cc:1048] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
W20260709 12:19:31.038398 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 74350us
I20260709 12:19:31.036762 19007 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:30.944185 25397 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:30.965770 25019 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:30.944059 25389 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.071841 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.000735 25369 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.944696 25201 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
W20260709 12:19:31.106222 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 69569.2us
I20260709 12:19:31.040117 25347 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:31.039347 25303 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.062026 25166 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.037429 25308 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:31.118902 18264 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:31.066669 25454 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.063777 25456 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:31.131875 19040 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
W20260709 12:19:31.132426 18412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.017s
W20260709 12:19:31.123415 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.016s	sys 0.016s
I20260709 12:19:31.029204 25285 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.137151 18198 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 11 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 13 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.141333 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:31.111392 25134 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:31.187796 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.017s	sys 0.017s
W20260709 12:19:31.246142 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 691478us
W20260709 12:19:31.215032 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 96230.9us
I20260709 12:19:31.122776 25347 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.247643 25347 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:31.101977 25141 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.157931 25457 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.252085 18415 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:31.188191 18412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.018s
I20260709 12:19:31.255434 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.139041 25166 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.257606 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.258517 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:31.272372 18442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.010s
W20260709 12:19:31.272881 18442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.009s	sys 0.010s
I20260709 12:19:31.116515 25369 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.277684 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.279150 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.279963 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.286360 19040 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:31.156859 25393 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 12:19:31.156328 25397 consensus_queue.cc:1059] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
W20260709 12:19:31.274219 25215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.001s	sys 0.030s
I20260709 12:19:31.222477 25461 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.137717 25303 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.302387 25461 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:31.302889 25461 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.303661 25303 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:31.296911 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 192904us
W20260709 12:19:31.263832 25411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.004s	sys 0.015s
I20260709 12:19:31.299715 25029 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:31.354148 25462 log.cc:927] Time spent T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:19:31.353909 25029 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.354288 25173 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:31.355681 25464 log.cc:927] Time spent T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:19:31.356354 25029 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.301568 25369 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.274175 25391 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:31.306614 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:31.297420 18373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.009s	sys 0.012s
W20260709 12:19:31.314069 19044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.010s	sys 0.015s
I20260709 12:19:31.358603 25391 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:31.295944 25215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.002s	sys 0.031s
W20260709 12:19:31.358999 19044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.010s	sys 0.016s
I20260709 12:19:31.359547 25391 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:31.367959 25486 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:31.368582 25391 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:31.371680 25411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.004s	sys 0.016s
I20260709 12:19:31.309675 25445 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:31.309871 25222 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.399446 25222 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.401885 25222 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.275327 25457 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:31.349139 25463 log.cc:927] Time spent T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 12:19:31.309742 25456 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.404438 25457 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.355064 25173 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:31.408190 25173 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:31.358944 18373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.010s	sys 0.013s
I20260709 12:19:31.409576 24987 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.410349 24987 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.413007 24987 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.375515 25162 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:31.380147 25389 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:31.286621 25125 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:31.388110 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 430773us
I20260709 12:19:31.415149 25125 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.415750 18266 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:31.417002 25190 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:31.417529 25125 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:31.342165 25217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.000s	sys 0.025s
I20260709 12:19:31.370931 25391 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.418746 25190 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:31.384223 25141 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:31.453033 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 146497us
I20260709 12:19:31.426025 25453 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.452636 24796 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.463771 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:31.455036 25433 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.505339 25433 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.462352 25457 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:31.507486 25457 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.488166 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.500264 25461 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:31.502457 25431 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.724s	user 0.005s	sys 0.006s
I20260709 12:19:31.481328 25453 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:31.503693 25285 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:31.454895 25445 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.456360 25222 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.477362 25454 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.506959 25190 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.466310 24933 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:31.508059 25457 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.516150 25460 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.529557 18298 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:31.529963 18298 raft_consensus.cc:614] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:31.478879 24796 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.516494 25345 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.520567 25217 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.545083 25019 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.517656 25431 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.548205 25433 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.565291 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.523272 25222 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.523887 25190 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.527653 25155 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.520748 25445 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.520557 25454 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.568109 25393 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.532756 25460 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.558324 25345 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.572943 25278 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.573103 25201 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
W20260709 12:19:31.606550 19043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.019s	sys 0.022s
I20260709 12:19:31.572732 25357 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:31.571758 25433 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.576308 25155 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.563282 25217 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.608155 24796 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.608639 25019 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:31.609229 25478 log.cc:927] Time spent T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:19:31.632373 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:31.619020 25201 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.634330 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.630532 25166 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.610311 25190 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.610661 25393 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.646247 18443 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:31.611208 25222 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.612448 25445 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.613874 25454 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:31.625315 19043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.020s	sys 0.022s
I20260709 12:19:31.618655 25345 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.608552 25347 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.615208 25431 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.618517 25460 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.647871 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.624111 25141 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.623996 25459 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.618785 25278 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.662083 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.690863 18444 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:31.660526 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.686753 19040 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:19:31.688114 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:31.644313 25162 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:31.643759 24796 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.709973 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:31.630422 25433 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.631578 25357 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.718590 25162 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:31.653097 25457 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.713500 25416 consensus_queue.cc:1059] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:19:31.743618 25217 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.706493 25459 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:31.744954 25482 log.cc:927] Time spent T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:19:31.745452 25457 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.772452 25459 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.807945 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:31.820377 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:31.822388 18369 raft_consensus.cc:2410] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:19:31.822281 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:31.823612 18415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.014s	sys 0.022s
W20260709 12:19:31.824355 18415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.015s	sys 0.023s
W20260709 12:19:31.824854 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 556497us
I20260709 12:19:31.823783 18373 raft_consensus.cc:2410] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:31.726261 24796 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.835810 24822 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.836386 18366 raft_consensus.cc:1240] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:31.836963 24822 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.708426 25141 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.857121 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:31.859881 25166 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:31.861687 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:31.736894 25201 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.915760 25503 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:31.918659 25393 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.919970 25347 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.922562 25278 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.929608 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:31.923449 25459 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:31.950857 18373 raft_consensus.cc:1240] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:19:31.957538 25401 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:31.756232 25453 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:31.970295 25433 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.982313 25459 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:31.985370 25459 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:31.984788 25453 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:31.986027 25483 log.cc:927] Time spent T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.370s	user 0.000s	sys 0.000s
I20260709 12:19:31.992019 25296 raft_consensus.cc:3055] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:31.992553 25296 raft_consensus.cc:740] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:31.993525 18291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 2948) took 1038 ms. Trace:
I20260709 12:19:31.995827 25155 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.996958 25141 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:31.994067 18291 rpcz_store.cc:276] 0709 12:19:30.955363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.956129 (+   766us) service_pool.cc:224] Handling call
0709 12:19:30.968103 (+ 11974us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:30.968455 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:30.968484 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:30.969113 (+   629us) log.cc:844] Serialized 52 byte log entry
0709 12:19:31.310559 (+341446us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:31.804232 (+493673us) log.cc:844] Serialized 14 byte log entry
0709 12:19:31.804799 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:31.805037 (+   238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:31.985128 (+180091us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:31.992867 (+  7739us) raft_consensus.cc:1685] finished
0709 12:19:31.993003 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:31.993219 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":355448,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":54919,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":450,"spinlock_wait_cycles":5163904,"thread_start_us":817458,"threads_started":2,"wal-append.queue_time_us":593714}
I20260709 12:19:31.994072 25296 consensus_queue.cc:260] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.001637 25296 raft_consensus.cc:3060] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:32.001924 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 175713us
I20260709 12:19:32.002211 24796 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.003352 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.003589 25493 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.004429 25493 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:31.961720 25401 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:31.963012 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.011865 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.014185 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.014403 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.014806 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:32.015069 18369 raft_consensus.cc:1240] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:19:32.016450 25493 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.017058 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.017753 18441 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:32.017714 25401 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.019855 18446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1653) took 1263 ms. Trace:
I20260709 12:19:32.019953 25357 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.020803 18371 raft_consensus.cc:2468] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 15.
I20260709 12:19:32.000507 24822 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.028237 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.029215 18445 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:32.020323 18446 rpcz_store.cc:276] 0709 12:19:30.756201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.756369 (+   168us) service_pool.cc:224] Handling call
0709 12:19:30.769729 (+ 13360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:30.769968 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:31.270524 (+500556us) log.cc:844] Serialized 14 byte log entry
0709 12:19:31.908587 (+638063us) log.cc:844] Serialized 14 byte log entry
0709 12:19:31.983165 (+ 74578us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18000bc260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d8f522 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd 0000000000388634 0000000000388479 00007f3c65d7283a
0709 12:19:32.001303 (+ 18138us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18000bc260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d09c0a 00007f3c6f449219 00007f3c6cd5058f 00007f3c65d8f571 00007f3c65d917c1 00007f3c65d91779 00007f3c65d91709 00007f3c65d916d1 00007f3c65d909cd
0709 12:19:32.019144 (+ 17841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:32.019272 (+   128us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:32.019397 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.019489 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.019633 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155767,"prepare.queue_time_us":621329,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":43851,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12393,"spinlock_wait_cycles":15154688,"thread_start_us":1030657,"threads_started":3,"wal-append.queue_time_us":487781}
I20260709 12:19:32.039695 25401 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.021232 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:32.040293 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 114211us
I20260709 12:19:32.043489 24933 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.043624 25503 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.044391 24933 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.044549 25503 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.045629 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.046314 18370 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:32.047514 25215 raft_consensus.cc:493] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:32.047693 18371 raft_consensus.cc:1240] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:32.048291 25215 raft_consensus.cc:515] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.050882 25215 leader_election.cc:290] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.046854 24933 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.046965 25503 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.062369 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.062132 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.069167 25411 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:32.069308 25166 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.070217 25166 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.071372 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.072538 18368 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 24: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:32.075941 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.076520 18373 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:32.077879 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.078593 18368 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:32.079730 25354 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.080425 25354 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.081156 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.081554 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.082362 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:32.082653 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.083137 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.083608 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.083806 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.084443 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.084784 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.085831 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.087074 25354 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.080444 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.092070 18372 raft_consensus.cc:2410] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:32.104899 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.106621 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:32.112197 19008 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.113687 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.116202 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.117022 18298 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:32.121913 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.122252 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.122434 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:32.122778 18295 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:32.123941 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.124357 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.125017 18293 raft_consensus.cc:2393] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 17: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:32.125571 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:32.126439 25296 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:32.126927 25296 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.127482 25296 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.128042 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.129593 25141 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.127499 18368 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:32.131129 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.131729 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:32.133086 25369 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:32.133610 25301 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:32.134083 25301 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.133841 25369 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.134518 25301 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.135301 25190 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:32.136071 25190 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.138510 25190 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:32.139823 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 130334us
I20260709 12:19:32.143581 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:32.144426 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 123348us
I20260709 12:19:32.144948 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.145201 18263 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.146164 18373 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:32.147217 25461 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.149379 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:32.145265 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.155571 25296 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.157970 25296 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.166810 25162 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.167119 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.160183 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.172894 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.178968 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.177973 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bc6be8433ea4f43a3c0ac6dfe27398e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.183538 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.184599 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.188038 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.189167 18299 raft_consensus.cc:2393] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 22: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:32.190105 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.190804 18299 raft_consensus.cc:3060] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:32.182600 24822 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.192111 24822 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.192633 24822 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.176010 24796 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:32.194103 24796 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.199517 19038 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:32.200038 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.200875 18292 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:32.201957 18369 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 18: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:32.202947 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.208650 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.209052 25166 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.209385 18295 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 23.
I20260709 12:19:32.209887 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.210409 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.211989 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.212697 18370 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.215936 25503 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.216727 25503 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.217136 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.217995 18368 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 23: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:32.218237 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.219079 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.219044 18300 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:32.219859 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.220855 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.220855 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.222617 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.222747 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.219838 18374 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:32.224210 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.224869 18373 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:32.224215 18300 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:32.211166 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.227671 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.228514 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.229655 25493 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:32.230494 25493 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.232708 25369 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.233292 25493 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.228982 18298 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:32.233662 25162 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.185619 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.235663 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.237077 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.239494 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.291467 18292 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:32.292414 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.239668 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.243597 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.294189 18296 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:19:32.179828 18263 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.250771 25503 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.251799 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:32.294801 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51584.7us
I20260709 12:19:32.254881 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.229183 18294 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:32.255963 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:32.256528 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.208160 24796 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.262068 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.263906 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.265308 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:32.266608 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.267729 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.269166 18370 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:32.271834 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.187055 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:32.330133 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 143150us
I20260709 12:19:32.272512 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.292490 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.244812 25391 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:32.252573 25301 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.297497 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:32.298293 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 138645us
I20260709 12:19:32.328665 19044 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:32.299377 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.310120 24822 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.311439 25401 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.312624 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.316814 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.339434 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.317919 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.319645 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.320147 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.320626 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:32.320740 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.321655 19037 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:32.322321 25453 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:32.344705 19038 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:32.344960 19035 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.345075 25453 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.346148 18373 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:32.346266 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:32.347256 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.347538 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.348174 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.348577 25453 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.344627 18367 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 14.
I20260709 12:19:32.351574 19043 raft_consensus.cc:2393] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 24: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.351826 19036 raft_consensus.cc:2393] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.352001 25301 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.352169 25411 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:32.352476 18295 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:32.352679 25411 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.353125 25411 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:32.353510 18339 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.16.193.67:40227 timed out after 1.589s (SENT) [suppressed 7 similar messages]
I20260709 12:19:32.322737 18368 raft_consensus.cc:1240] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:19:32.323841 18299 raft_consensus.cc:2468] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:32.322888 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.331733 18292 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:32.344444 25125 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:32.328574 25493 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:32.361529 19042 raft_consensus.cc:2393] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.361328 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:32.361902 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:32.366653 24796 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:32.367022 25369 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:32.367360 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.367676 24796 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.368007 18368 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.368181 24796 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.371981 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.373086 25453 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.361342 25503 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:32.375788 25503 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.378332 25503 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.378983 24822 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.383538 19039 raft_consensus.cc:2393] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.385178 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.386049 19041 raft_consensus.cc:2393] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:32.387034 25467 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:32.387722 25467 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.390195 25467 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.392549 25495 consensus_queue.cc:1059] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:32.393836 25495 raft_consensus.cc:3055] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:32.393556 25278 consensus_queue.cc:1059] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:32.394208 25495 raft_consensus.cc:740] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:32.394644 25278 raft_consensus.cc:3055] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:32.395306 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.367525 25369 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.400810 18297 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:32.410988 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.412068 25503 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:32.412576 25503 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.413025 25503 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:32.415290 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.415362 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:32.417593 18371 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.420459 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:32.395103 25278 raft_consensus.cc:740] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:32.425076 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.415828 25125 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.429852 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:32.404698 25369 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.425932 19039 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:32.432138 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.434974 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.435484 25125 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.439275 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.442124 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.446028 19044 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:32.405725 25495 consensus_queue.cc:260] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.455615 25278 consensus_queue.cc:260] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.439615 25453 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:32.456771 25278 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:32.461977 18291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:32.462764 18291 raft_consensus.cc:606] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:32.463373 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:32.471664 25401 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:32.472502 25401 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:32.479914 18368 raft_consensus.cc:2468] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:32.481082 25504 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:32.482149 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4d1763576e4c49849611360f37665a" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.488471 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.460018 25453 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.450114 25411 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.550668 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.555635 18368 raft_consensus.cc:1240] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:32.560989 24933 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.561566 24933 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.564582 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:32.565728 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 143188us
I20260709 12:19:32.566555 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:32.567387 25461 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:32.568071 25357 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:32.580067 25166 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:32.580686 25166 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.582952 25166 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.593736 25495 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:32.548985 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 64223.9us
I20260709 12:19:32.594918 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.601101 25433 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:32.601655 25433 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.602161 25433 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:32.609843 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.610834 25391 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:32.611017 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 126502us
I20260709 12:19:32.613078 25493 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:32.597693 25125 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.544078 25453 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:32.631633 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.636899 25125 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.641053 25411 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.653992 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:32.653798 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 300424us
I20260709 12:19:32.670054 19044 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 22.
W20260709 12:19:32.672123 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 105695us
I20260709 12:19:32.673981 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.554643 24796 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.674849 24933 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:32.652091 25433 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.727828 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.731731 18371 raft_consensus.cc:2468] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:32.687829 24933 consensus_queue.cc:1059] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:32.749423 24796 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:32.755972 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63719.7us
I20260709 12:19:32.745297 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:32.776225 25354 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.777005 25354 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.779944 25354 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:32.784025 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 170393us
W20260709 12:19:32.788379 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 248766us
I20260709 12:19:32.789381 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.791666 25125 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:32.792065 25125 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:32.792415 25125 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.794399 25453 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.794382 25401 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.797642 25215 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.809042 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:32.806428 25354 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:32.819639 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.804286 25481 consensus_queue.cc:1059] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:19:32.830163 25369 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.832463 25369 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:32.836620 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.837379 18263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.842875 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.840255 25401 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:32.824102 25215 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.852295 19009 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:32.853188 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.853441 25354 raft_consensus.cc:2804] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:32.854171 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 47967.6us
I20260709 12:19:32.857384 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.859843 19009 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:32.870886 25530 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:19:32.851192 25369 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.876490 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
W20260709 12:19:32.879894 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 285081us
I20260709 12:19:32.876550 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.855806 24822 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.855979 25354 raft_consensus.cc:697] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:32.872092 25433 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.884761 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:32.888394 18412 leader_election.cc:304] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:32.888756 19006 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:32.874075 25296 raft_consensus.cc:2804] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:32.888715 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:32.873492 25495 consensus_queue.cc:1059] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:19:32.883751 25453 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:32.884501 25411 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:32.906356 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:32.898397 25539 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.914163 25190 raft_consensus.cc:3055] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:32.902683 25296 raft_consensus.cc:697] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:32.914315 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:32.915707 25503 consensus_meta.cc:287] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.502s	user 0.014s	sys 0.000s
I20260709 12:19:32.911692 18338 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.908425 24822 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:32.951264 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 69488.9us
I20260709 12:19:32.940157 25190 raft_consensus.cc:740] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:32.938710 25125 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:32.959430 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 51287.2us
I20260709 12:19:32.961491 25461 raft_consensus.cc:3055] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:32.961951 25461 raft_consensus.cc:740] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:32.943359 25503 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:32.964731 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:32.975158 18338 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.979857 25296 consensus_queue.cc:237] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:32.977851 25540 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:32.980005 25285 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.992766 25540 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:32.993237 25540 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:32.998327 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84109.3us
I20260709 12:19:32.953210 25539 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.000751 18338 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:32.978951 25401 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:32.952417 25354 consensus_queue.cc:237] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.002279 25538 raft_consensus.cc:2804] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:33.002774 25215 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.007016 25538 raft_consensus.cc:697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:33.007478 25125 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:32.979743 25190 consensus_queue.cc:260] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.008054 25538 consensus_queue.cc:237] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.008638 25190 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:33.011673 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 54957us
I20260709 12:19:33.013795 25401 raft_consensus.cc:993] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:33.017529 25539 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:33.011423 25453 raft_consensus.cc:2764] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:33.020920 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.013955 18266 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:33.023573 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 135297us
I20260709 12:19:33.027474 25401 raft_consensus.cc:1081] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:33.028330 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:33.028833 25215 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.029258 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:33.030118 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:33.032371 25389 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.032639 25369 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:33.033139 25369 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:33.033555 25369 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.033614 25278 raft_consensus.cc:3055] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:33.034174 25278 raft_consensus.cc:740] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:33.048985 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:33.067441 18374 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:33.068501 18370 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:33.076169 25550 raft_consensus.cc:493] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:33.076967 25550 raft_consensus.cc:515] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.083560 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:33.084421 18366 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:33.086757 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.086903 18367 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:33.024809 25411 raft_consensus.cc:2749] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:33.088506 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:33.090523 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.092974 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:33.093588 18374 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:19:33.093891 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 57447.6us
I20260709 12:19:33.094038 25481 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.094477 25481 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:33.094853 25481 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.095690 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.096261 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:33.096859 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84691.2us
I20260709 12:19:33.097235 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:33.043877 25459 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:33.085285 25545 log.cc:927] Time spent T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:19:33.118758 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.122539 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.122912 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.123196 18368 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:33.123613 19042 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:33.122936 19044 raft_consensus.cc:1240] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:33.091060 24933 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.126216 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:33.035115 25278 consensus_queue.cc:260] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.109807 25125 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:33.130941 18338 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.121126 25389 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.127481 25357 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.140436 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.141947 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.141399 25556 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.142661 18299 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:33.144629 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.129084 25539 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:33.132813 25550 leader_election.cc:290] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.118876 25459 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.154299 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.155026 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.160880 18295 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:33.155663 25504 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:33.164417 18299 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:33.145269 25453 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:33.099466 25461 consensus_queue.cc:260] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.142292 25411 consensus_queue.cc:1059] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:19:33.181485 25369 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.169588 25453 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:33.183699 25453 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.174950 25503 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.133014 24933 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.178653 25354 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.160771 25357 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.160353 25278 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.203516 18292 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:33.173887 25539 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:33.208609 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 89960.3us
I20260709 12:19:33.210567 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:33.191748 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 44039.7us
I20260709 12:19:33.198202 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.194082 25552 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:33.201619 19040 raft_consensus.cc:1240] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:33.212694 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.214016 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.215314 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.216076 19044 raft_consensus.cc:1240] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:19:33.212208 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.218703 18263 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:33.220036 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.223564 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.217540 25389 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.224726 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.204957 24933 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.209689 25369 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:33.237946 19039 raft_consensus.cc:2468] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 23.
W20260709 12:19:33.237742 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 140565us
I20260709 12:19:33.236706 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.248062 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:33.254083 19042 raft_consensus.cc:1240] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:33.255647 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46054.4us
W20260709 12:19:33.261006 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 48870.9us
I20260709 12:19:33.268056 18338 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:33.242291 25503 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:33.210748 25354 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.254294 24933 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:33.277789 18444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:33.278440 18444 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:33.278771 18444 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.257854 25461 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:33.279408 18266 leader_election.cc:304] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.266835 25459 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.289836 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.282514 24796 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:33.296159 24933 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:33.303167 24933 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:33.308485 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 85814us
I20260709 12:19:33.255132 25552 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.288411 25504 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.315656 18295 raft_consensus.cc:2468] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:33.314311 18374 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:33.266149 25481 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.296695 25495 raft_consensus.cc:2749] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:33.326726 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 97592.2us
W20260709 12:19:33.331192 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54208.6us
I20260709 12:19:33.321947 25504 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:33.339694 18297 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 15: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:33.348816 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 86655.2us
I20260709 12:19:33.367857 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.371729 18338 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.377269 24796 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.380330 25554 consensus_queue.cc:1059] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:33.381412 25554 raft_consensus.cc:3055] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:33.381799 25554 raft_consensus.cc:740] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.350796 25504 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.384274 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 85760.9us
I20260709 12:19:33.388018 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.387761 25554 consensus_queue.cc:260] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.391505 25554 raft_consensus.cc:3060] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.391989 25559 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:33.391438 25560 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:33.374372 25553 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:33.401479 25411 raft_consensus.cc:3055] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:33.401440 25553 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.402061 25411 raft_consensus.cc:740] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.403185 25411 consensus_queue.cc:260] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.388813 25389 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:19:33.408286 18412 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:33.395190 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 66650.7us
I20260709 12:19:33.420153 18339 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.427928 19042 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:19:33.509817 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 121854us
W20260709 12:19:33.515903 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 92158.1us
I20260709 12:19:33.412292 25557 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.425460 25369 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:19:33.412252 25552 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.416196 25354 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:33.424218 25539 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.451388 25166 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:33.520668 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 123042us
W20260709 12:19:33.411042 25563 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:19:33.552783 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.525784 18445 raft_consensus.cc:1240] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:33.524554 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 116353us
I20260709 12:19:33.556234 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.411159 25555 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:33.556057 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 136023us
I20260709 12:19:33.525100 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:33.526760 18338 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.427904 25551 consensus_queue.cc:1059] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 12:19:33.562335 18338 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.411599 25481 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.404084 25553 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.524570 18444 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:33.564950 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 54018.1us
I20260709 12:19:33.566985 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.551905 25538 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:19:33.571555 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46634.5us
I20260709 12:19:33.572129 18415 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:33.575184 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.577781 18444 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.579995 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:33.565407 25301 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:33.588164 25166 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:33.588629 25553 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:33.589061 25166 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:33.589622 25166 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:33.590111 25166 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.591629 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:33.592566 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 163889us
I20260709 12:19:33.593128 25559 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:33.593931 25559 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.593302 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:33.563659 25552 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:33.565239 25433 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:33.617228 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.568994 25503 raft_consensus.cc:2804] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:33.619688 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.628660 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.625787 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.630523 19044 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:33.632400 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.593940 25296 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.589105 25553 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.629328 18298 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:33.637568 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.638819 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:33.643298 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 72084.3us
I20260709 12:19:33.614495 25552 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.618705 25555 raft_consensus.cc:2804] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:33.645179 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.617086 25285 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:33.649905 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.622831 25354 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.652714 18300 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:19:33.637966 25540 consensus_meta.cc:287] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.000s
I20260709 12:19:33.653820 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.631395 25433 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:33.656175 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.656438 18294 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:33.660737 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.661339 18300 raft_consensus.cc:3060] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:33.665138 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.638242 25503 raft_consensus.cc:697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:33.666991 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.667829 18293 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:33.673275 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.675953 18296 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:33.676120 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:33.679929 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104805us
I20260709 12:19:33.653602 25285 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.689971 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.669235 25555 raft_consensus.cc:697] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.664283 25540 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.697751 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.694571 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:33.701678 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 50936.8us
I20260709 12:19:33.702840 25584 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:33.703886 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.704633 18296 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:33.698694 25495 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:33.716739 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.695302 25569 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.683358 25553 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.723170 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:33.723875 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 157235us
I20260709 12:19:33.726065 18338 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.727564 18374 raft_consensus.cc:1240] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:33.728806 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.715735 25495 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.718871 25571 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.735203 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:33.735698 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.734051 25495 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.740411 18412 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.741703 18412 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.743371 25586 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.744123 25586 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.744048 25493 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:33.744992 25586 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.751204 25588 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.737603 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.754344 18293 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:33.754909 18412 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.731429 25560 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.759635 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.711844 25552 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.773525 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.775059 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.775394 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.776759 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.777416 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.755918 25285 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:33.758237 25369 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.717643 25433 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:33.789750 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.791015 18412 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.791936 25539 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:33.792413 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.793282 25539 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.798779 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.797905 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.723724 25503 consensus_queue.cc:237] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:33.803409 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 192691us
I20260709 12:19:33.808408 18443 raft_consensus.cc:2393] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 19: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:33.809039 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.769856 25584 consensus_queue.cc:1059] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:19:33.810622 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.811674 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.813709 25589 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:33.813875 25590 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.814409 25590 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:33.814805 25590 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.815152 25278 consensus_meta.cc:287] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.004s
I20260709 12:19:33.815430 25553 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:33.816120 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:33.817350 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.818440 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:33.818953 18292 raft_consensus.cc:3055] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:33.819265 18292 raft_consensus.cc:740] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:33.820014 18292 consensus_queue.cc:260] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.820902 18292 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.770210 25555 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.771975 25560 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.823523 25540 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.825856 25560 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.829406 18292 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
W20260709 12:19:33.829838 25461 consensus_meta.cc:287] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 12:19:33.832510 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d266f68528964b1981ffe1df49861b08" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:33.833657 18447 raft_consensus.cc:2393] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 23: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
W20260709 12:19:33.833954 25453 consensus_meta.cc:287] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.650s	user 0.002s	sys 0.008s
I20260709 12:19:33.834408 25453 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.836051 25166 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.837551 25495 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.838086 25166 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:33.758410 25504 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.847218 18412 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:33.848402 18300 raft_consensus.cc:2468] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:33.857015 18300 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:33.858450 18412 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:33.858457 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:33.865576 18374 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:33.835114 25551 raft_consensus.cc:3055] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:33.870694 18371 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.889945 18296 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:33.896400 25369 raft_consensus.cc:3055] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:33.896860 25369 raft_consensus.cc:740] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.897703 25369 consensus_queue.cc:260] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:33.902206 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 43863.6us
I20260709 12:19:33.903438 25549 raft_consensus.cc:2804] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:33.904100 25582 consensus_queue.cc:1059] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:19:33.911561 25538 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:33.912339 25538 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:33.918278 25590 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:33.923360 25495 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:33.924852 25190 consensus_meta.cc:287] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: Time spent flushing consensus metadata: real 0.889s	user 0.012s	sys 0.000s
I20260709 12:19:33.874190 25554 raft_consensus.cc:2804] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:33.928571 25453 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:33.939404 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 248036us
I20260709 12:19:33.872376 25551 raft_consensus.cc:740] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.899067 25369 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:33.953445 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.964586 18339 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:33.915108 25538 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:33.976748 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.979311 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.976282 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.981529 18368 raft_consensus.cc:1240] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:33.982419 18372 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:33.983340 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:33.939671 25549 raft_consensus.cc:697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.943534 25453 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:33.991779 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:33.992738 19040 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:33.994545 25411 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.995689 19040 raft_consensus.cc:1240] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:33.997459 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b135a527cce4b14b789d23618d411b0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.999186 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.999739 25584 consensus_queue.cc:1059] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:34.001585 25559 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.001951 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.963944 25554 raft_consensus.cc:697] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:33.954860 25389 consensus_queue.cc:1059] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:19:34.004645 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:33.986274 25539 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.009088 25539 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.005946 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:34.010421 19040 raft_consensus.cc:3060] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.978331 25540 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:34.003717 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:34.018570 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 165363us
I20260709 12:19:34.014729 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:34.020340 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 303803us
I20260709 12:19:34.022439 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.024507 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:34.026288 18366 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 15: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:34.027436 18412 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:34.028527 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.013368 25586 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:34.026630 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.030630 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.006251 25559 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.006659 25504 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.016121 25453 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.007470 25554 consensus_queue.cc:237] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.066180 25453 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:34.051857 18412 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.053802 25294 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:34.054315 25584 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:34.028438 25582 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.070935 19040 raft_consensus.cc:2468] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:34.028635 25415 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:34.083449 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 91748.3us
W20260709 12:19:34.084779 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.049849 24796 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.049213 25555 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:34.005606 25551 consensus_queue.cc:260] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.029076 25553 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:19:34.072660 25539 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:34.111482 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57979.2us
I20260709 12:19:34.007979 25549 consensus_queue.cc:237] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.115634 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:34.116140 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 64364.6us
I20260709 12:19:34.114604 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.116954 18412 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.115909 18338 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.116271 18373 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:34.118857 25555 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.123734 25553 consensus_queue.cc:1243] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer 991dcd2a70c74fdb89ca9494cc09fb7b log is divergent from this leader: its last log entry 19.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:19:34.139310 25590 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:34.090469 25582 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.327672 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:34.191964 25166 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.316141 25611 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:34.317178 25604 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.257s	user 0.000s	sys 0.009s
I20260709 12:19:34.318076 18367 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:34.318162 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.319109 19040 raft_consensus.cc:1240] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:34.339619 18372 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 21: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:34.319991 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.320642 25550 raft_consensus.cc:3055] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:19:34.321309 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 206797us
I20260709 12:19:34.168326 25583 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:34.121557 25504 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:19:34.123890 25540 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:34.343288 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:34.111619 25584 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.176513 25606 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.324963 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:34.322851 25587 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.589s	user 0.003s	sys 0.010s
I20260709 12:19:34.332109 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.332808 25582 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.332227 25166 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.341823 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.345813 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:34.354679 19041 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:34.340550 25550 raft_consensus.cc:740] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:34.362957 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:34.353456 25540 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:34.349086 24796 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.369160 19035 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:19:34.351670 25503 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.353232 25551 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:34.368206 18339 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.355621 25611 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.374354 18369 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:34.364637 25590 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:34.378432 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 346509us
I20260709 12:19:34.383468 18370 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:34.387873 18372 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:34.388784 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.376623 25453 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.404284 25554 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.405009 25433 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:34.404857 25554 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.407058 25554 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:34.407895 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 65165.8us
I20260709 12:19:34.408062 25540 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.411403 25586 raft_consensus.cc:3055] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:34.411922 25586 raft_consensus.cc:740] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:34.387327 25550 consensus_queue.cc:260] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.413571 25550 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.378087 24796 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.414769 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:34.387012 25577 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.415856 24796 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.415782 25577 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.417987 25577 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.419549 25613 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.420173 25613 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.420698 25582 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.420665 25613 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.421365 25582 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.424361 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.424817 25391 consensus_queue.cc:1059] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.425103 24796 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:34.381242 25599 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:19:34.426292 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:34.426635 25585 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.706s	user 0.000s	sys 0.002s
I20260709 12:19:34.428654 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.429353 25504 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.381347 25459 consensus_queue.cc:1059] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:19:34.383860 25481 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.431169 25481 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.432701 25559 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:34.430894 25569 raft_consensus.cc:3055] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:34.433197 25481 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.433228 25569 raft_consensus.cc:740] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:34.432310 18266 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:34.434352 25569 consensus_queue.cc:260] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.435755 18266 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:34.436733 25401 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:34.423072 25613 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:34.430645 18339 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.421406 18442 raft_consensus.cc:2410] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:34.435534 25569 raft_consensus.cc:3060] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.442198 25559 consensus_queue.cc:1059] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 12:19:34.445824 18264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.448624 25559 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:34.455096 25559 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:34.455857 25559 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.457005 25559 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.459043 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.447628 25590 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:34.449836 25552 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:34.463052 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:34.454041 18446 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:19:34.463683 18446 pending_rounds.cc:85] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 15
I20260709 12:19:34.463907 18446 pending_rounds.cc:107] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:19:34.464167 18446 raft_consensus.cc:2889] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:34.382648 25611 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.442428 25190 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:34.465293 18300 raft_consensus.cc:1275] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:19:34.466614 25611 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:34.467919 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:34.468498 18300 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:34.471319 25555 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:34.388434 25584 consensus_queue.cc:1059] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 12:19:34.475255 25401 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:34.475667 25401 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.476029 25401 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.477444 25166 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:34.376293 25503 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.479612 25503 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:34.481628 18444 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 15.
I20260709 12:19:34.482445 25582 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:34.483726 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:34.486404 25580 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.792s	user 0.000s	sys 0.003s
I20260709 12:19:34.490842 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:34.492892 19044 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:34.497587 18292 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: []
W20260709 12:19:34.508626 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.509756 25504 raft_consensus.cc:3055] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:34.510228 25504 raft_consensus.cc:740] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:34.511780 18300 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:34.521289 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.521991 18293 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:34.524936 18292 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:34.525024 25586 consensus_queue.cc:260] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.511961 18294 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:34.529146 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:34.511080 25504 consensus_queue.cc:260] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.549331 25504 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:34.520766 25401 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.560374 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:34.564815 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.571718 18444 raft_consensus.cc:2410] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:34.573398 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.579218 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.580082 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.580715 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.581678 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:34.585687 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.595979 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.596655 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.596951 19041 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:34.597267 18296 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:34.601078 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.601876 18374 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:34.605525 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.607399 18300 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:34.608696 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.609544 18294 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:34.610378 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.610499 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.611186 18296 raft_consensus.cc:2410] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:34.611142 18300 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:34.613062 18299 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:34.614189 18442 raft_consensus.cc:1275] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:34.615769 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.617552 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.620601 18295 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:34.620802 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.621874 19044 raft_consensus.cc:2393] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 20: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:34.623190 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.623834 18375 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:34.612946 25481 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
W20260709 12:19:34.624917 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 178569us
I20260709 12:19:34.626173 18339 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.612898 25555 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:34.627398 25538 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:34.627978 24796 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:34.628094 25538 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.628552 24796 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.630738 24796 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:34.632144 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.637666 18339 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.640848 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.644402 18266 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.651300 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:34.651631 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:34.652331 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.652961 19038 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:34.662483 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:34.577795 25552 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:19:34.675757 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.679545 19042 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:34.582501 25569 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:19:34.682430 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.616513 25582 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:19:34.636686 25415 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:34.690852 19036 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:19:34.619911 25503 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:19:34.652843 25595 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:19:34.699831 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 65863.8us
I20260709 12:19:34.681708 25555 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:34.701843 25555 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.702242 25555 raft_consensus.cc:3060] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:34.647929 25433 consensus_queue.cc:1059] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:19:34.663393 25584 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:34.710906 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 64995.3us
I20260709 12:19:34.696141 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.709152 25411 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:19:34.660840 25162 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.681885 25459 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:19:34.733855 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 352117us
I20260709 12:19:34.721249 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.688639 25495 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:34.737439 19040 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:34.693547 25415 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.737342 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:34.739168 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.739395 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:34.768546 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 67438.6us
I20260709 12:19:34.716899 25190 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:34.771756 18449 raft_consensus.cc:1240] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:34.804013 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 64759.1us
I20260709 12:19:34.742201 25215 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:34.697788 24796 consensus_queue.cc:1059] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:19:34.718384 25584 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:34.818331 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 47374us
I20260709 12:19:34.812580 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:34.813429 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 177353us
I20260709 12:19:34.744267 25567 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:34.738839 25495 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.825969 18339 leader_election.cc:304] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:34.716709 25590 consensus_queue.cc:1048] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 12:19:34.801407 25190 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.730669 25354 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:19:34.806577 25369 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:34.810542 25453 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:34.742611 25415 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.733603 25162 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.742821 25613 consensus_queue.cc:1059] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:19:34.844983 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:34.830515 25584 raft_consensus.cc:3060] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.743010 25611 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:19:34.862960 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.846318 25190 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.862082 18373 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:34.862063 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:34.857384 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 116257us
I20260709 12:19:34.851186 25433 raft_consensus.cc:3055] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:34.841567 25495 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:34.855584 25401 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:34.868937 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 70110us
I20260709 12:19:34.870630 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.870961 18339 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.869131 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:34.858847 24796 raft_consensus.cc:2749] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:34.876684 25162 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:34.883526 25620 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:34.884382 25493 raft_consensus.cc:3055] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:34.954934 25620 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:34.955204 25493 raft_consensus.cc:740] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:34.955595 25620 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.956554 25493 consensus_queue.cc:260] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.957921 25493 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.886688 25559 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:34.888485 25586 raft_consensus.cc:3060] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.904500 25554 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:34.963446 25554 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:34.964566 25554 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.966462 25554 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.972810 25620 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.974309 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:34.975067 18374 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.975713 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.905534 25555 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:34.911327 25552 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.864377 25369 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:34.913605 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.914149 25553 consensus_queue.cc:1059] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:34.915095 19044 raft_consensus.cc:1240] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:34.916067 19040 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:19:34.917402 25540 raft_consensus.cc:3055] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:34.863395 25615 consensus_queue.cc:1048] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:19:34.925575 18368 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:34.927143 25467 raft_consensus.cc:3055] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:34.851001 25538 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:19:34.927798 19036 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:34.988011 25549 raft_consensus.cc:493] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.929382 18367 raft_consensus.cc:1275] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:19:34.933739 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.936661 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:35.012734 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 76181.7us
I20260709 12:19:35.014007 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:34.939620 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 68724.7us
I20260709 12:19:35.015672 18339 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:35.018523 19009 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:35.019284 25608 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:35.019407 25582 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:34.996212 24796 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:35.020598 25582 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:35.021159 24796 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.021965 25582 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:35.023033 18198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 214) took 53742 ms (client timeout 15000 ms). Trace:
W20260709 12:19:35.023423 18198 rpcz_store.cc:269] 0709 12:18:41.280110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.282615 (+  2505us) service_pool.cc:224] Handling call
0709 12:18:41.406517 (+123902us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b54003ebda8. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6f06e331 00007f3c6f0693fc 00007f3c6ec3ae84 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:35.022825 (+53616308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:31.765645 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:31.766374 (+   729us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:31.766418 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:19:31.766444 (+    26us) tablet.cc:662] Decoding operations
0709 12:19:31.766921 (+   477us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:31.766994 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:31.767029 (+    35us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:19:31.767434 (+   405us) tablet.cc:701] Row locks acquired
0709 12:19:31.767459 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:19:31.767633 (+   174us) write_op.cc:270] Start()
0709 12:19:31.767771 (+   138us) write_op.cc:276] Timestamp: P: 1783599571767578 usec, L: 0
0709 12:19:31.767798 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:19:31.768617 (+   819us) log.cc:844] Serialized 919 byte log entry
0709 12:19:32.190886 (+422269us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5400030820. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68df36b8 00007f3c68df32fa 00007f3c68e77965 00007f3c68e77e09 00007f3c6c8b2581 00007f3c6c8b1d58 00007f3c6c8b69f1 00007f3c6c8b69a9 00007f3c6c8b6939 00007f3c6c8b6901
0709 12:19:32.622704 (+431818us) op_driver.cc:464] REPLICATION: finished
0709 12:19:32.931419 (+308715us) write_op.cc:301] APPLY: starting
0709 12:19:32.931527 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:32.933783 (+  2256us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:32.933829 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:32.935215 (+  1386us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:32.964401 (+ 29186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:19:32.973691 (+  9290us) write_op.cc:312] APPLY: finished
0709 12:19:33.024096 (+ 50405us) log.cc:844] Serialized 33 byte log entry
0709 12:19:33.024329 (+   233us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:33.024464 (+   135us) write_op.cc:454] Released schema lock
0709 12:19:33.024537 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:33.079941 (+ 55404us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:33.079972 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31,"reported_tablets":37,"spinlock_wait_cycles":13132672,"child_traces":[["op",{"apply.queue_time_us":286056,"apply.run_cpu_time_us":5885,"apply.run_wall_time_us":148541,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12479,"num_ops":2,"prepare.queue_time_us":461296,"prepare.run_cpu_time_us":38469,"prepare.run_wall_time_us":433747,"raft.queue_time_us":467585,"raft.run_cpu_time_us":55792,"raft.run_wall_time_us":200875,"replication_time_us":854626,"spinlock_wait_cycles":38957824,"thread_start_us":1357365,"threads_started":4,"wal-append.queue_time_us":207251,"wal-append.run_cpu_time_us":101992,"wal-append.run_wall_time_us":2050811}]]}
I20260709 12:19:34.949249 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:34.992523 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 78996.7us
I20260709 12:19:35.015009 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:34.998944 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:35.036640 19037 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.001356 18375 raft_consensus.cc:1240] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:34.869717 25433 raft_consensus.cc:740] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:35.009387 18296 raft_consensus.cc:1275] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:35.027475 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:34.996413 25569 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:35.031919 18370 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:19:35.032382 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:35.033186 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 45440.7us
I20260709 12:19:35.010278 25606 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:35.040185 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.041086 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:35.041956 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:35.043037 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:34.940223 25503 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:35.050915 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:34.999838 25540 raft_consensus.cc:740] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:35.044691 18374 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:35.051532 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.046253 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.003008 25467 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:35.056407 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:35.056496 18266 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:35.069168 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:34.996506 25552 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.072273 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.043984 25550 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:35.077096 18298 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:35.080746 18292 raft_consensus.cc:1240] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:35.082636 18444 raft_consensus.cc:2468] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 23.
I20260709 12:19:35.070338 25503 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:35.043362 25549 raft_consensus.cc:515] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.084003 25609 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.084367 25504 consensus_meta.cc:287] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.003s
I20260709 12:19:35.106220 18444 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:35.054152 25190 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.101878 18441 raft_consensus.cc:1275] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:35.089543 25559 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:35.077987 25569 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
W20260709 12:19:35.119334 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 56252.1us
I20260709 12:19:35.077360 25162 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:19:35.124790 25467 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.126279 25467 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.132876 25369 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.140429 25552 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:35.141427 25555 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:35.142354 25415 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.144804 25415 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:35.145418 19042 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:35.146088 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:35.146692 25495 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.147114 18373 raft_consensus.cc:2393] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 19: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:35.148514 19037 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:35.149634 25503 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.080051 25584 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.158658 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:35.163517 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 75743.1us
I20260709 12:19:35.163894 18339 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:35.112747 18195 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.165104 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.165877 18412 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.166821 25369 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:35.168327 18300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.168941 18300 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 24.
I20260709 12:19:35.155508 25559 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:35.169250 25538 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:35.157401 25555 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:19:35.117012 25609 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.161602 25552 consensus_queue.cc:1048] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:19:35.165077 18195 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 23, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 23 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.173529 18298 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:35.208981 18339 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.185385 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:35.211074 18339 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:35.209110 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:35.212370 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.212755 18370 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:35.212842 18339 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.214555 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:35.215484 18296 raft_consensus.cc:2368] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
I20260709 12:19:35.191844 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:35.216617 19039 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:35.170411 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:35.090220 25433 consensus_queue.cc:260] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.195410 18195 catalog_manager.cc:5697] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 20 to 21, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 21 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.233183 18446 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:35.173684 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:35.201314 18442 raft_consensus.cc:2410] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:19:35.201591 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
W20260709 12:19:35.263204 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92874.4us
I20260709 12:19:35.263780 18195 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 8 to 18, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 18 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.264710 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:35.266491 18367 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:35.202419 18445 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:35.268194 18448 raft_consensus.cc:1275] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:35.186265 25495 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.211967 25538 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:35.219373 25504 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:35.179558 25411 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.280268 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:35.180671 25190 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.180948 25549 leader_election.cc:290] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.266971 18195 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 15, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.196875 25620 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.257696 25599 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.287602 18195 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 18, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 18 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.193447 25584 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:35.278972 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 66215.9us
I20260709 12:19:35.181969 25559 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.291034 18195 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 16, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 16 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.173229 25638 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:35.266261 25459 raft_consensus.cc:2804] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:35.299943 25459 raft_consensus.cc:697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:35.300238 18292 raft_consensus.cc:2468] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:35.300940 25459 consensus_queue.cc:237] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.189020 25609 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:35.269759 25493 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:19:35.316634 19039 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:35.294550 18195 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 17, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 17 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.318914 18195 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 20, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 20 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.321044 18412 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:35.331498 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.337235 18295 raft_consensus.cc:2468] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:35.339648 18370 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:35.322463 18195 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 18, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 18 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:35.342537 25586 raft_consensus.cc:2804] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:35.343149 25586 raft_consensus.cc:697] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:35.343077 18195 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 18, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 18 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.284744 25411 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.344271 25586 consensus_queue.cc:237] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.318558 25493 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.318288 25613 raft_consensus.cc:3055] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:35.353631 18195 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 16, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.352645 18412 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:35.293767 25538 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:19:35.360695 25411 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.356247 25493 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:35.358029 25549 raft_consensus.cc:2804] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:35.365280 25549 raft_consensus.cc:697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:35.290529 25599 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.368103 25549 consensus_queue.cc:237] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.369063 25599 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:35.370648 18367 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:35.299464 25504 consensus_queue.cc:1048] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:19:35.371330 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:35.372093 19039 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:35.356244 18195 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 8 to 10, term changed from 11 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to <none>, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.377789 18195 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.376267 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:35.380882 18195 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 18, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.382980 25467 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.383463 25467 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.383803 25467 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:35.386549 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 94502.5us
I20260709 12:19:35.356370 25613 raft_consensus.cc:740] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:35.390686 18195 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 16, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.392380 18339 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:35.392050 18195 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 20, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 20 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.416826 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.417174 18195 catalog_manager.cc:5697] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 21. New cstate: current_term: 21 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.376000 24796 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:35.422603 18263 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:35.418556 18195 catalog_manager.cc:5697] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 18, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.430975 19006 leader_election.cc:400] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
I20260709 12:19:35.431725 19006 leader_election.cc:403] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:19:35.429723 18195 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 16, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.433934 18195 catalog_manager.cc:5697] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.414377 25599 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:35.460337 18195 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 15, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.431321 25481 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:35.437840 25415 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:35.476117 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:35.480716 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.462956 18195 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 15, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 15 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.519810 18195 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:35.517800 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:35.521468 18195 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 16, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 16 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.523561 18291 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:19:35.522974 18195 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 6 to 8, term changed from 10 to 17, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.524554 18195 catalog_manager.cc:5697] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 22, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.459980 25504 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:35.526017 18195 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 22, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:35.537940 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.542681 18292 raft_consensus.cc:2410] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:35.512149 25565 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.539345 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.512976 25481 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:35.477048 25599 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:35.548693 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:35.551057 19040 raft_consensus.cc:2368] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:35.545324 18195 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:19:35.561882 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 73394.1us
I20260709 12:19:35.531454 25550 consensus_queue.cc:1048] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:19:35.548151 25638 raft_consensus.cc:493] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:35.582017 18338 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.605638 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1717) took 1041 ms. Trace:
I20260709 12:19:35.544080 25504 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.551014 25565 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.606071 18448 rpcz_store.cc:276] 0709 12:19:34.563968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.564090 (+   122us) service_pool.cc:224] Handling call
0709 12:19:34.598434 (+ 34344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.268680 (+670246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.605427 (+336747us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400258fd4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7ce01 00007f3c68e7b83e 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6
0709 12:19:35.605584 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531460864}
I20260709 12:19:35.601056 25638 raft_consensus.cc:515] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } attrs { replace: true } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:35.615008 25648 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:35.613330 25481 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.651759 25613 consensus_queue.cc:260] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.658047 25550 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:35.658864 25550 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.661294 25550 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.616902 25459 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:35.676352 25493 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.686116 25504 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.693252 25569 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:35.693473 25596 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.693857 25569 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.696677 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.702152 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.621868 24796 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:19:35.638404 25599 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.705404 25125 consensus_queue.cc:1048] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:35.704214 25294 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:35.693991 25596 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.772646 25596 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:35.773138 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:35.774272 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:35.777299 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 159043us
I20260709 12:19:35.780408 25589 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:35.782061 25493 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:35.766430 25467 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.785156 25467 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.764585 25569 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:35.768271 25638 leader_election.cc:290] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:35.770228 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:35.644021 25640 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:35.785862 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:35.788403 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:35.809038 19035 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.763209 25565 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:35.785879 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:35.794026 25415 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 19. Current term is 20.
I20260709 12:19:35.807668 18444 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 24. Current term is 25. Ops: []
I20260709 12:19:35.831908 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:35.850665 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.839032 25589 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.852080 25589 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:35.854420 25589 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:35.854753 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:35.856801 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:35.857542 18368 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:35.860647 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:35.861364 18298 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:35.867885 18294 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:19:35.868808 18415 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.966995 19035 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:35.841858 25640 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:35.986294 25569 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.987028 18413 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.987427 25467 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:35.968612 18412 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:35.988535 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 112550us
I20260709 12:19:35.967715 25538 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:35.984767 25596 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.015728 18266 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.034188 18446 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:19:36.035877 25638 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:36.007973 25550 raft_consensus.cc:2804] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:36.040777 25550 raft_consensus.cc:697] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:35.997267 25569 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.041692 25550 consensus_queue.cc:237] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.043696 25569 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.009241 25467 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.044685 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.045364 19036 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:36.046022 25640 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.046350 25467 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.047601 18412 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:36.049757 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.052709 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:36.065966 18375 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:36.049149 25481 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:36.055953 25586 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:36.080164 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.097214 18294 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:36.063782 25461 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.099267 25481 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:36.100493 25481 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.075856 25586 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.103158 25586 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.060570 25569 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.107765 25569 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.110416 25569 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.065122 25493 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:36.028640 25538 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.064029 25579 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.098610 25461 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.032927 25596 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.068320 25640 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.168394 25495 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.168290 18182 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e23c0ce8d89940da8eac6a6002e349c5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:36.169178 18182 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 95519.2us
I20260709 12:19:36.169059 25495 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.171746 25596 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:36.151794 25640 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.175024 25640 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.135040 25579 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:36.184013 18338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.000s	sys 0.085s
W20260709 12:19:36.184331 18338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.085s
W20260709 12:19:36.184589 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 580611us
I20260709 12:19:36.185906 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.186635 18373 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
W20260709 12:19:36.189639 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 124117us
I20260709 12:19:36.194530 18338 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:36.201061 18368 raft_consensus.cc:1240] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:36.201155 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.203415 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.205837 18373 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:36.206739 19043 raft_consensus.cc:1240] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:36.192065 25495 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:36.213671 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.215703 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.179903 25540 consensus_queue.cc:260] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.217296 25540 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:36.219970 25461 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.225219 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.206944 25503 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:36.232800 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.231594 25481 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:36.236474 25481 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.238481 25481 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.239315 25467 raft_consensus.cc:493] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.239862 25467 raft_consensus.cc:515] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.241966 25467 leader_election.cc:290] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:36.242837 25666 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:36.244912 25433 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:36.247073 25653 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.645s	user 0.000s	sys 0.016s
I20260709 12:19:36.249719 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:36.252140 25657 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.649s	user 0.016s	sys 0.042s
I20260709 12:19:36.259336 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.260041 18299 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:36.262550 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.263171 18375 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:36.197173 25549 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:19:36.264523 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.265275 18368 raft_consensus.cc:2393] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:36.266384 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48158 (request call id 3121) took 1214 ms. Trace:
I20260709 12:19:36.195312 25504 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:19:36.266775 18444 rpcz_store.cc:276] 0709 12:19:35.052484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.119443 (+ 66959us) service_pool.cc:224] Handling call
0709 12:19:35.807209 (+687766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.838772 (+ 31563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.263128 (+424356us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400258ad0. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e7b8ad 00007f3c6de40419 00007f3c673b7bd4 00007f3c673b7b62 00007f3c673b7a91 00007f3c673b7a0c 00007f3c673b6cc2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:36.266299 (+  3171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495961856}
I20260709 12:19:36.272100 25579 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.274093 25538 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.233814 25669 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:36.275653 25669 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.277717 25669 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.280078 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.282994 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:36.284957 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.285832 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.286209 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5742413fd242b88ca8c4668e6b8144" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.287354 18298 raft_consensus.cc:2468] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 25.
I20260709 12:19:36.288201 18373 raft_consensus.cc:2393] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 21: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:36.288985 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.289893 18298 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:36.293538 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.294070 18368 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:36.292452 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.297549 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.230175 25611 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.295014 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:36.230628 25461 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.298141 18371 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:36.295959 18373 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:36.296504 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.304200 25285 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:36.305001 25481 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:36.210780 25550 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:19:36.297451 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:36.290028 25669 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.309134 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 189172us
I20260709 12:19:36.310456 18375 raft_consensus.cc:2393] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:36.312119 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.311350 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.318007 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0931a54c049145dfad672173c0b29a40" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.318543 18367 raft_consensus.cc:3060] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.313443 18412 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:36.295117 25540 raft_consensus.cc:493] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.322225 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.307677 25285 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.323268 25285 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.326839 25549 raft_consensus.cc:2804] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:19:36.327435 25549 raft_consensus.cc:697] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:36.328114 25540 raft_consensus.cc:515] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.328250 25549 consensus_queue.cc:237] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.329622 18374 raft_consensus.cc:1275] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:36.310653 25461 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.344165 18367 raft_consensus.cc:2468] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:36.345481 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.346662 18368 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:36.347798 19040 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:36.350497 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.350608 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.320078 25493 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:36.321063 25616 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:36.350925 18266 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:36.295715 25586 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.319908 25354 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:36.384657 18338 leader_election.cc:400] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:36.385260 18338 leader_election.cc:403] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:36.360447 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35973fcee55042b284c6f62e2681c304" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.365787 19036 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.372121 18415 leader_election.cc:304] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
W20260709 12:19:36.226780 25599 consensus_queue.cc:1175] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e23c0ce8d89940da8eac6a6002e349c5" responder_term: 21 status { last_received { term: 20 index: 15 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:36.319868 25567 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:19:36.320349 25669 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.358778 25616 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.310246 25611 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.385627 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
W20260709 12:19:36.390169 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.393707 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.394308 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.396283 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:36.355202 25493 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.393635 25620 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f for earlier term 20. Current term is 21.
I20260709 12:19:36.412052 18415 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.406840 18338 leader_election.cc:304] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.418147 19036 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:36.418326 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.386880 25550 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.419900 25550 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.420282 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.420495 25495 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:36.420925 25495 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.421269 25495 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.422176 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:36.388311 25565 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:36.433320 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.395558 25569 raft_consensus.cc:2749] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:36.436926 19036 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:36.414000 25616 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:36.490650 18441 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:36.424103 25493 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.413655 25554 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:36.422842 25550 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:36.539077 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 127087us
I20260709 12:19:36.429741 25620 raft_consensus.cc:2749] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:36.542428 18415 leader_election.cc:304] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.435329 25565 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.395258 25354 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.427999 25674 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.116s
W20260709 12:19:36.568885 25677 log.cc:927] Time spent T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:19:36.532464 25461 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.434813 25503 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
W20260709 12:19:36.591358 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49013us
I20260709 12:19:36.593406 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.594513 18374 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:36.596613 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.598266 19035 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:36.553184 25565 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:36.545415 25683 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:36.573161 25669 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.552947 25682 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:36.575450 25620 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.581491 25640 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.581121 25590 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:36.635412 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.580925 25549 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.599709 25569 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.642885 18338 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.612829 25554 raft_consensus.cc:2749] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:36.652386 18412 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:36.653522 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.654142 18374 raft_consensus.cc:3060] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.670744 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:36.660200 25638 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:36.684413 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:36.640702 25620 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:36.685845 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 381317us
I20260709 12:19:36.641376 25669 consensus_queue.cc:1048] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:19:36.664041 25683 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:36.640198 25640 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.648559 25590 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.702553 18374 raft_consensus.cc:2468] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:36.655344 25549 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.652679 25569 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.705885 25596 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.712427 25596 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.713263 25620 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.714521 25596 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.716776 25549 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:36.718547 25640 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:36.720664 18412 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.720978 25611 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:36.728505 25354 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.732681 25590 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:36.735390 25679 log.cc:927] Time spent T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:19:36.736281 25676 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:19:36.745798 25616 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.720702 25569 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:36.681489 25586 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.732535 25285 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.748790 25493 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.749835 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.757403 18371 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:36.750680 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.758059 18367 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:36.753177 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.751355 25215 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.761777 19039 raft_consensus.cc:2368] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:36.763500 25215 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.765946 25215 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.767683 19037 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 20: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:36.750936 25495 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:36.769881 25687 log.cc:927] Time spent T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:36.770596 25495 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:36.772709 19036 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:36.775866 25285 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:36.776911 25671 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.462s	user 0.003s	sys 0.005s
W20260709 12:19:36.779054 25673 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.461s	user 0.002s	sys 0.005s
I20260709 12:19:36.785784 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.787532 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.788547 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:36.789055 18366 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.785219 25669 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:19:36.805805 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.806413 18294 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:36.806269 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.807862 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.811888 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.812808 18292 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:36.817718 25689 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:19:36.818581 25586 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.819185 18296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45180 (request call id 3002) took 1302 ms. Trace:
I20260709 12:19:36.819548 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.819694 25682 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.820245 25682 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.819576 18296 rpcz_store.cc:276] 0709 12:19:35.517199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.517448 (+   249us) service_pool.cc:224] Handling call
0709 12:19:36.818964 (+1301516us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54001cd9d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:36.819102 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152720640}
I20260709 12:19:36.819193 25586 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.803835 25688 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.822387 25688 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.823936 25549 raft_consensus.cc:493] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:36.824520 25688 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.824442 25549 raft_consensus.cc:515] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.793923 25674 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:19:36.825103 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.825280 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.825696 18296 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:36.825825 18292 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:36.826462 25549 leader_election.cc:290] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:36.827493 18366 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:36.827571 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.828317 18444 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 19: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:36.830507 18368 raft_consensus.cc:1275] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:36.832783 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.833101 18294 raft_consensus.cc:1240] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:19:36.833352 18440 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 16.
I20260709 12:19:36.802382 25215 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:19:36.841078 18266 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:36.841321 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.841609 18375 raft_consensus.cc:3055] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:19:36.841940 18375 raft_consensus.cc:740] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:36.843392 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.844543 18300 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:36.845498 18415 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:36.846369 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:36.846699 18299 raft_consensus.cc:1240] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:36.851255 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.851758 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8903d70080d24236be955433a92791c7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.851958 18445 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:36.852332 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.832019 25565 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.848397 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:36.842711 18375 consensus_queue.cc:260] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:36.863481 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.863708 18375 raft_consensus.cc:3060] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:36.864785 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:36.865186 25644 log.cc:927] Time spent T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 1.297s	user 0.012s	sys 0.009s
I20260709 12:19:36.865464 19042 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:36.864985 25550 raft_consensus.cc:2804] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:36.866003 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.866173 25550 raft_consensus.cc:697] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:36.866496 19041 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59422 (request call id 1263) took 1596 ms. Trace:
I20260709 12:19:36.866652 18449 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:36.867209 25550 consensus_queue.cc:237] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.867647 18447 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:36.867590 18291 raft_consensus.cc:3060] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:36.885463 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:36.864785 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:36.875145 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.887053 18298 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:36.892252 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.860975 25674 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:36.922000 18264 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:36.871989 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:36.923353 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:36.923983 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52038.9us
I20260709 12:19:36.866907 19041 rpcz_store.cc:276] 0709 12:19:35.269601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.269721 (+   120us) service_pool.cc:224] Handling call
0709 12:19:35.277106 (+  7385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.277553 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.277595 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.279857 (+  2262us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.448026 (+168169us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:35.448065 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.448127 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.864597 (+1416470us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:36.866179 (+  1582us) raft_consensus.cc:1685] finished
0709 12:19:36.866198 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.866404 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4145,"spinlock_wait_cycles":28060928,"thread_start_us":155500,"threads_started":1,"wal-append.queue_time_us":228869}
W20260709 12:19:36.924700 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.899101 25554 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:36.942852 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:36.943610 18296 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:36.944495 25610 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:36.945032 25610 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:36.951968 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:36.953231 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 152398us
W20260709 12:19:36.955416 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 70023.9us
I20260709 12:19:36.956250 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:36.957211 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:36.957576 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.952755 18440 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:36.930132 25538 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:36.964797 18292 raft_consensus.cc:2393] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:36.970542 18375 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:19:36.972692 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:36.976706 18375 pending_rounds.cc:85] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 14
I20260709 12:19:36.976999 18375 pending_rounds.cc:107] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:19:36.977237 18375 raft_consensus.cc:2889] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:36.942458 25554 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.008136 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:36.987507 19044 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:36.936890 25686 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:37.006755 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:36.947165 25610 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:36.932477 25638 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:19:36.975499 25538 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.025113 25569 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:37.025540 25569 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.025902 25569 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.013566 25554 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.015731 25686 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:37.027781 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 104340us
W20260709 12:19:37.032886 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 66828.8us
I20260709 12:19:37.017989 25616 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:37.034842 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53961us
I20260709 12:19:37.036504 18263 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:37.037870 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:37.038736 18264 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:37.027647 25538 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.024156 25493 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.042088 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:37.044943 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 110529us
I20260709 12:19:37.045800 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:37.037000 25674 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:37.036460 25686 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.062171 25685 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:37.083766 25685 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:37.063975 25493 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:19:37.068089 25692 log.cc:927] Time spent T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:37.069160 25701 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:37.033478 25354 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:19:37.120775 25354 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:37.121131 25354 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:37.122128 25354 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.076894 25565 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:37.087139 18338 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.132786 18373 raft_consensus.cc:1275] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:37.133605 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.133913 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.134593 18371 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 23.
I20260709 12:19:37.136929 18263 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:37.148360 25711 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:37.093415 25684 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.158413 18415 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:37.093962 25559 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.159574 25559 raft_consensus.cc:493] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:37.160143 25559 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.095800 25685 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.050664 25549 consensus_queue.cc:1048] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:19:37.161250 25686 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.111739 25674 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.164942 25686 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.169778 25684 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:37.149439 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 110757us
I20260709 12:19:37.171068 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:37.171643 18264 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:37.173111 25624 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:37.177330 25559 raft_consensus.cc:515] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.177762 25538 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.181246 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:37.181928 19039 raft_consensus.cc:2368] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:37.190798 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.191638 18367 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.090689 18296 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:37.107132 18291 raft_consensus.cc:1275] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:19:37.197687 19006 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:37.208339 18263 leader_election.cc:400] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:37.204757 25215 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:37.235081 18264 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:37.217505 25616 raft_consensus.cc:2804] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:37.232939 18263 leader_election.cc:400] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:37.240120 18263 leader_election.cc:403] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:19:37.240142 18367 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:37.205665 25569 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.207500 25493 consensus_queue.cc:1048] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:19:37.247130 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:37.247517 25702 consensus_queue.cc:1048] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:37.241811 18415 leader_election.cc:304] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:37.232146 25706 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:37.245113 25701 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.251977 25549 raft_consensus.cc:2804] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:37.277180 25586 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:37.278822 19009 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.291132 25613 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:37.292349 25538 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.264771 25559 leader_election.cc:290] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:37.296468 25495 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:37.300212 25554 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.302814 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:37.373879 19037 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.255048 25711 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.316579 25540 leader_election.cc:290] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:37.320700 25718 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:37.319080 25682 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.380868 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:37.379056 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 147292us
W20260709 12:19:37.324867 25715 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:37.382588 18448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:37.329955 18441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48158 (request call id 3131) took 1044 ms. Trace:
I20260709 12:19:37.338177 25569 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.340939 25354 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:37.314798 25707 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:37.263720 25616 raft_consensus.cc:697] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:37.364493 18368 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:37.365211 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.380563 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:37.390604 18298 raft_consensus.cc:3060] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.383329 18441 rpcz_store.cc:276] 0709 12:19:36.285447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.285569 (+   122us) service_pool.cc:224] Handling call
0709 12:19:37.329705 (+1044136us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54003b10d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:37.329866 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348152192}
I20260709 12:19:37.359648 25549 raft_consensus.cc:697] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:37.390663 18367 raft_consensus.cc:2393] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 19: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:37.323494 25703 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821) with higher term. Message: Invalid argument: T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 21. Current term is 22.
I20260709 12:19:37.392005 18447 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 16: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:37.377463 25674 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:37.266714 25706 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:37.391812 25707 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
W20260709 12:19:37.439481 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 73030.5us
I20260709 12:19:37.447474 18298 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
W20260709 12:19:37.450870 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 565246us
I20260709 12:19:37.398978 25590 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.459075 25682 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:37.453212 19037 raft_consensus.cc:2468] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 22.
I20260709 12:19:37.460729 18442 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:37.428186 25674 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.472505 25658 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.473033 25658 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.474489 25674 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:37.474886 25658 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.475389 25704 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:37.494288 25704 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.496480 25704 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.475867 25591 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:37.386392 25711 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.478060 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:37.497905 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:37.476537 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:37.479187 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:37.499343 25711 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.455892 25707 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:37.485965 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:37.468747 25590 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.504485 25590 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.507432 25707 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:37.508216 25708 log.cc:927] Time spent T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:19:37.509932 25720 log.cc:927] Time spent T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:19:37.511040 25707 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.531551 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.533036 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.538244 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e8da9e5fd9449f93381781e125fada" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.542179 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.543174 18367 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
W20260709 12:19:37.545581 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63595.4us
I20260709 12:19:37.546360 18338 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.546540 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.549012 18373 raft_consensus.cc:2368] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
W20260709 12:19:37.554217 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 43617.4us
I20260709 12:19:37.556845 18370 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:37.562925 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6021c985b4e44fe880350a321132a9b5" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.564011 18371 raft_consensus.cc:2393] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:37.566509 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:37.566905 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4688e522b8b48f7bb98e0832680f9e2" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:37.567531 18296 raft_consensus.cc:3060] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.569895 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:37.593910 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.596736 18373 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:37.582216 18296 raft_consensus.cc:2468] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:37.565611 25638 raft_consensus.cc:493] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:37.579105 25599 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:37.575035 25611 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:37.615010 18368 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:37.575752 25704 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:37.626772 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.633597 18373 raft_consensus.cc:3060] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.512159 25549 consensus_queue.cc:237] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.588527 25569 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:37.555622 25616 consensus_queue.cc:237] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.646785 18266 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:37.650096 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 103902us
I20260709 12:19:37.635746 25554 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.638018 25682 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:37.660354 18449 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:37.662300 25683 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.663002 25683 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.664423 18444 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:37.659981 25682 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.674134 19009 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.617679 25638 raft_consensus.cc:515] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.652488 25701 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.635416 25611 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.639019 25704 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.651064 18338 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:37.682324 25724 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:37.682951 25724 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:37.684289 25682 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.685276 25724 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.685856 25643 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:37.686395 25643 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.642609 25569 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:37.695286 18266 leader_election.cc:400] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
I20260709 12:19:37.695751 18266 leader_election.cc:403] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:37.698920 18368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:37.699774 18368 raft_consensus.cc:2410] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 12:19:37.700357 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 49252.8us
I20260709 12:19:37.699442 19009 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:37.708276 18412 leader_election.cc:304] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.701545 25190 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:19:37.703433 25658 raft_consensus.cc:2804] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:37.704646 25294 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.732760 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.733498 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:37.722956 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.789415 19038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:37.772614 25613 raft_consensus.cc:2749] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:37.790107 25643 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:37.790431 18373 raft_consensus.cc:2468] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:37.773167 25725 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:19:37.790565 18412 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:37.806967 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 98903.5us
I20260709 12:19:37.791369 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:37.688342 25706 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.792388 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:37.794617 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.816732 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.813509 25611 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.814507 25569 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:37.808467 18294 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
W20260709 12:19:37.818810 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 84383.1us
I20260709 12:19:37.816759 19043 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:37.701900 25693 raft_consensus.cc:2749] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 for earlier term 16. Current term is 17.
I20260709 12:19:37.824023 18372 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:37.828379 18412 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:37.841408 18292 raft_consensus.cc:1275] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:19:37.842092 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.842576 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.843829 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.844995 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.846423 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:37.847092 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:37.820528 25658 raft_consensus.cc:697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:37.851213 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.837555 25301 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:37.922386 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 71277.3us
I20260709 12:19:37.927309 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:37.930557 25301 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
W20260709 12:19:37.934120 25733 log.cc:927] Time spent T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:19:37.934332 25301 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:37.934420 25726 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:37.938637 19042 raft_consensus.cc:1275] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:19:37.939497 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 92459.2us
I20260709 12:19:37.941341 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.941466 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:37.941903 25736 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:37.942454 25736 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.942893 25736 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:37.955384 25696 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:19:37.837253 25569 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:37.960680 25569 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.961189 25569 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.964434 25704 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:37.965330 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:37.847622 25722 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:19:37.845623 25611 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.942237 25643 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:37.928325 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 77194.1us
I20260709 12:19:37.984818 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:37.990742 19039 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:37.932452 25613 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.999822 18441 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:37.993357 25611 consensus_queue.cc:1048] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:19:37.982640 25125 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.019788 18449 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:37.922077 25354 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.022989 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.020936 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.024309 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b81b910b60b0422f9db4c0164608fcbb" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.006569 25301 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.009622 25682 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:38.044790 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.013187 25739 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.048738 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.021823 25554 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:38.071910 25729 log.cc:927] Time spent T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:19:38.072214 25732 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:19:37.982129 25658 consensus_queue.cc:237] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.060158 25738 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:38.029829 25354 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.057229 25682 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.085602 25682 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.061535 25554 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.088008 25554 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.087795 25736 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.089929 25736 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.048002 25638 leader_election.cc:290] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.095353 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:38.096086 18295 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:38.023429 25125 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.061030 25709 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:38.115849 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.083101 25354 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.124541 25301 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.125500 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:38.133179 18374 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:38.128141 18447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 3117) took 1090 ms. Trace:
I20260709 12:19:38.133525 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.138367 18446 raft_consensus.cc:1240] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:38.133661 18447 rpcz_store.cc:276] 0709 12:19:37.037654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.037810 (+   156us) service_pool.cc:224] Handling call
0709 12:19:37.170925 (+133115us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b58001a0798. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d57a22 00007f3c6ec708d9 00007f3c6ddcaee0 00007f3c6de940ab 00007f3c6de94131 00007f3c6de420a0 00007f3c6de41984 00007f3c673bc6d4 00007f3c673bc662
0709 12:19:38.127892 (+956967us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54003b24d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:38.128049 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503185408}
I20260709 12:19:38.139565 19009 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:38.139876 19009 leader_election.cc:403] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:38.139715 18295 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:38.141762 18338 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:38.142791 25599 raft_consensus.cc:2804] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:38.143440 25599 raft_consensus.cc:697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:38.144557 25599 consensus_queue.cc:237] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.152944 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.157477 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:19:38.183884 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 44444.2us
I20260709 12:19:38.131889 25709 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.192059 25704 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.194638 24822 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.206951 25728 raft_consensus.cc:3060] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:38.197211 25294 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.211432 24822 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.229252 25125 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.234318 25728 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:38.236758 24822 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.237888 18372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.221928 25354 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.242412 25354 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.250628 19043 raft_consensus.cc:2468] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:38.259471 25569 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.262506 25569 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.210394 25704 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.282249 25354 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.284281 25704 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.285575 25709 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.286633 18368 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:19:38.288177 25749 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:38.289274 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.289988 18449 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:38.290731 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.315120 18444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1744) took 1426 ms. Trace:
I20260709 12:19:38.315570 18444 rpcz_store.cc:276] 0709 12:19:36.888513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.888632 (+   119us) service_pool.cc:224] Handling call
0709 12:19:38.314953 (+1426321us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54003557d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:38.315056 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702565120}
I20260709 12:19:38.297122 25609 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
W20260709 12:19:38.322765 25669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.007s	sys 0.039s
W20260709 12:19:38.325280 25669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.008s	sys 0.040s
I20260709 12:19:38.357045 25591 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:38.363961 25704 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:38.364593 25704 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.366834 25704 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.357559 25591 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.367388 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.368547 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.369971 25559 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.370522 25559 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.372112 25549 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.372638 25549 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.373246 25559 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.375063 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.375159 25549 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.377295 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.377918 19037 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:38.377367 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.380451 19035 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 21: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:38.396034 25579 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:38.396600 25579 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.397085 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.397693 18443 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:38.398666 25579 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.402969 18449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.403700 18449 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:38.405269 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.405850 18373 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:38.406703 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.410802 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.411087 18370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.411340 25683 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.412375 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d788f8182ca4b13bddd2acdf511873f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.413393 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.414268 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:38.414750 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.414995 19039 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.416368 18366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.417397 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.418522 25595 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:38.418700 25467 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:38.418919 25595 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:38.419306 25467 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:38.419535 25569 consensus_queue.cc:1048] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:38.419872 25595 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.420250 25467 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.414331 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.422057 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.428077 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.461342 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.421108 25595 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:38.477795 19042 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:38.478125 25591 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.481405 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:38.482040 25591 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:38.490340 18442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 3125) took 1243 ms (client timeout 1594 ms). Trace:
W20260709 12:19:38.490756 18442 rpcz_store.cc:269] 0709 12:19:37.246770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.246897 (+   127us) service_pool.cc:224] Handling call
0709 12:19:38.490171 (+1243274us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54001851d4. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:38.490276 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486567296}
W20260709 12:19:38.491707 25762 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:19:38.492558 25504 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:38.482729 25726 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.484053 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:38.493624 25726 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.495718 25726 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.492725 18339 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.496869 18339 leader_election.cc:304] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.497524 25453 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.497720 25550 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:38.498117 25550 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.498171 25453 raft_consensus.cc:2749] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:38.498557 25550 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:38.488246 18448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48170 (request call id 3126) took 1106 ms. Trace:
I20260709 12:19:38.499356 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.500092 25453 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.499379 18448 rpcz_store.cc:276] 0709 12:19:37.382191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.382357 (+   166us) service_pool.cc:224] Handling call
0709 12:19:38.488079 (+1105722us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b5400424354. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de5571 00007f3c68e802fa 00007f3c6de41b17 00007f3c673bc6d4 00007f3c673bc662 00007f3c673bc591 00007f3c673bc50c 00007f3c673bb7c2 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:38.488191 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107726208}
I20260709 12:19:38.501297 18339 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.501920 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "353cbd244a354830a4e903a3b1be2b89" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.502017 25453 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:38.505617 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 81336us
I20260709 12:19:38.523497 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.523774 18296 raft_consensus.cc:1275] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:19:38.516629 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.570420 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:38.580474 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63930.2us
I20260709 12:19:38.543743 19007 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.581238 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.565308 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.538286 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.507326 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:38.506850 18264 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.524252 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.583513 18373 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.507262 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.582926 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.517076 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adea31c273544d04b7f16f9b9a3ae610" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:38.585983 18298 raft_consensus.cc:2368] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:38.546607 18339 leader_election.cc:304] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.508224 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.514770 19039 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:38.588307 25706 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:38.589370 25493 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.589663 25731 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:38.565953 25610 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:38.584446 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:38.598783 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 92038.4us
I20260709 12:19:38.506083 25550 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.593348 25682 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.610570 18263 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.597597 25724 raft_consensus.cc:2749] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.600128 25731 raft_consensus.cc:493] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:38.616783 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 58099.1us
W20260709 12:19:38.634003 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 49620us
I20260709 12:19:38.616314 25675 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.615896 25579 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.506362 25574 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:38.648319 18263 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.506145 25453 consensus_queue.cc:1059] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:19:38.648594 25731 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:38.645812 25591 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:38.649770 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 51227.4us
I20260709 12:19:38.644333 25711 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:38.662926 19043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0931a54c049145dfad672173c0b29a40"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:38.644236 25504 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.669042 19043 raft_consensus.cc:606] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:38.674243 19043 raft_consensus.cc:614] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:38.675804 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.644443 25704 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:38.680673 25682 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.681344 18297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45180 (request call id 3024) took 1227 ms. Trace:
I20260709 12:19:38.696811 18297 rpcz_store.cc:276] 0709 12:19:37.454163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.454290 (+   127us) service_pool.cc:224] Handling call
0709 12:19:37.454685 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:37.476530 (+ 21845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.476560 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:37.478935 (+  2375us) log.cc:844] Serialized 26 byte log entry
0709 12:19:38.326809 (+847874us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b1800488260. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec64281 00007f3c65d71e07 00007f3c6ebcc9a9 00007f3c65d8e69b 00007f3c65d8c93f 00007f3c65d8ec02 00007f3c68a1620b 00007f3c68a1bd23 00007f3c68a1c0d1 00007f3c68e37a0f
0709 12:19:38.567511 (+240702us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:38.567761 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.567934 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.567997 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.593157 (+ 25160us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b48001f2858. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6dda5aa1 00007f3c6dd9cb37 00007f3c6dd9c9ca 00007f3c6de9426f 00007f3c6de93f37 00007f3c6dea855c 00007f3c6dea8508 00007f3c6dea8488 00007f3c6dea8443 00007f3c6dea76ec 00007f3c6cdd1a96
0709 12:19:38.681105 (+ 87948us) raft_consensus.cc:1685] finished
0709 12:19:38.681130 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.681258 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21443,"prepare.queue_time_us":626,"prepare.run_cpu_time_us":1385,"prepare.run_wall_time_us":3800,"raft.queue_time_us":28085,"raft.run_cpu_time_us":1967,"raft.run_wall_time_us":25421,"spinlock_wait_cycles":13343360,"thread_start_us":240151,"threads_started":1,"wal-append.queue_time_us":1152527}
W20260709 12:19:38.685671 25766 log.cc:927] Time spent T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.128s	user 0.001s	sys 0.006s
I20260709 12:19:38.694190 25550 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.658823 25574 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.705698 25574 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.677166 25503 raft_consensus.cc:3055] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:38.706681 25503 raft_consensus.cc:740] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:38.673673 25703 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:38.632577 25610 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.716825 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:38.670131 25591 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.736166 25503 consensus_queue.cc:260] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:38.677881 25504 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.742098 25504 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:38.730439 25591 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.748643 18373 raft_consensus.cc:2468] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:38.750268 18415 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:38.775596 25467 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:38.786634 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:19:38.792407 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 145323us
W20260709 12:19:38.793330 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146045us
I20260709 12:19:38.791282 25675 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.794327 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:38.785902 25467 raft_consensus.cc:697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:38.801594 18266 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:38.795413 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 141317us
I20260709 12:19:38.803344 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.803903 25559 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.804949 25467 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.802676 25675 raft_consensus.cc:2804] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:38.817775 25675 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.818238 25675 raft_consensus.cc:3060] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:38.830085 18263 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:38.833067 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.842547 18373 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:38.839857 25709 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:38.846179 25709 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:38.846535 25709 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:38.847394 25574 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.853740 18295 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:19:38.855858 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 52325.3us
I20260709 12:19:38.856855 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.858798 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.866025 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.870379 25503 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.870383 25627 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.871989 25627 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.873915 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:38.874130 19039 raft_consensus.cc:1275] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:19:38.801020 25731 raft_consensus.cc:515] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.870522 25190 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:19:38.881243 19037 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 22: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:38.875334 19035 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:19:38.875602 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 193491us
I20260709 12:19:38.876078 18266 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.876336 19038 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:38.885483 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.886813 25190 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.891538 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.891803 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:38.892607 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.893643 25725 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:38.894048 18297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.894184 25725 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:38.895147 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9934c84956844d20a6258c5dca3833c6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.895814 18375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:33256
I20260709 12:19:38.896338 25725 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.877354 25610 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:38.896358 18375 raft_consensus.cc:606] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:38.897665 18375 raft_consensus.cc:614] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:38.898744 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:38.903052 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.875185 19007 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.914265 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:38.918031 18296 raft_consensus.cc:1275] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:38.937654 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.876637 25627 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:38.942183 18298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.944799 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.942571 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.943434 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fafcfc4614304a81b0eadc8f445bfdb8" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.944233 19006 leader_election.cc:400] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21.
I20260709 12:19:38.948480 18412 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.922082 25609 raft_consensus.cc:2749] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.953747 18296 raft_consensus.cc:1275] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:38.951555 19006 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.956857 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.957279 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.960681 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.961728 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:38.962215 19035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e62e9fad92b449d9315da4aa77082b4"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:38.962798 19035 raft_consensus.cc:606] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:38.963030 19035 raft_consensus.cc:614] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:38.952056 24796 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.972889 18412 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:38.954864 25411 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:38.949430 25704 raft_consensus.cc:493] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:38.940222 25724 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:19:38.959911 25627 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.961668 25669 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.997494 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:38.946897 25726 consensus_queue.cc:1048] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:19:38.965504 25550 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.965145 25595 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.965963 25609 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:38.957672 25549 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:19:38.985667 25586 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:38.962924 25538 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:19:39.000355 25495 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:39.070747 25722 raft_consensus.cc:493] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.071355 25722 raft_consensus.cc:515] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.074066 25709 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:39.074566 25722 leader_election.cc:290] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.039733 25682 consensus_queue.cc:1048] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:19:38.994403 25704 raft_consensus.cc:515] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.047554 25702 consensus_queue.cc:1048] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:19:39.055483 25495 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:39.088634 25595 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:39.089344 25495 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:39.090046 25703 raft_consensus.cc:2749] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:39.091818 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 84242.5us
I20260709 12:19:39.092000 25731 leader_election.cc:290] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:39.092538 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.093264 25481 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.095150 18295 raft_consensus.cc:1240] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:39.097407 18291 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:39.098254 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:39.098809 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:39.099548 18446 raft_consensus.cc:2393] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 22: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:39.099941 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:39.070645 25675 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.107563 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:39.116878 18299 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
W20260709 12:19:39.120051 25782 log.cc:927] Time spent T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: Append to log took a long time: real 0.110s	user 0.002s	sys 0.000s
I20260709 12:19:39.110364 25599 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:39.121490 25599 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.123764 25599 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.123628 25591 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.124998 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:39.089151 25595 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.128005 25595 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:39.132793 25709 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:39.079105 25704 leader_election.cc:290] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.135628 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:39.112646 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:39.139320 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:39.139508 18195 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 298) took 6888 ms. Trace:
I20260709 12:19:39.141614 25591 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:39.141506 18195 rpcz_store.cc:276] 0709 12:19:32.251673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.252064 (+   391us) service_pool.cc:224] Handling call
0709 12:19:39.139431 (+6887367us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:36.234016 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:36.265400 (+ 31384us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:36.265449 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:19:36.265467 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:36.321370 (+ 55903us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:36.353253 (+ 31883us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:36.355094 (+  1841us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:19:36.747666 (+392572us) tablet.cc:701] Row locks acquired
0709 12:19:36.747716 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:19:36.747902 (+   186us) write_op.cc:270] Start()
0709 12:19:36.748052 (+   150us) write_op.cc:276] Timestamp: P: 1783599576747836 usec, L: 0
0709 12:19:36.748077 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:19:36.749091 (+  1014us) log.cc:844] Serialized 11471 byte log entry
0709 12:19:37.235591 (+486500us) op_driver.cc:464] REPLICATION: finished
0709 12:19:37.375977 (+140386us) write_op.cc:301] APPLY: starting
0709 12:19:37.378632 (+  2655us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:38.239089 (+860457us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:38.239187 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:38.681746 (+442559us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:38.682134 (+   388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:19:38.682185 (+    51us) write_op.cc:312] APPLY: finished
0709 12:19:38.682868 (+   683us) log.cc:844] Serialized 252 byte log entry
0709 12:19:38.799471 (+116603us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:38.802398 (+  2927us) write_op.cc:454] Released schema lock
0709 12:19:38.831161 (+ 28763us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:38.882185 (+ 51024us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:38.883438 (+  1253us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3532,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":140094,"apply.run_cpu_time_us":49492,"apply.run_wall_time_us":1468420,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125231,"num_ops":29,"prepare.queue_time_us":158196,"prepare.run_cpu_time_us":26008,"prepare.run_wall_time_us":601833,"raft.queue_time_us":302824,"raft.run_cpu_time_us":10149,"raft.run_wall_time_us":27545,"replication_time_us":487331,"thread_start_us":576212,"threads_started":5,"wal-append.queue_time_us":179818,"wal-append.run_cpu_time_us":20418,"wal-append.run_wall_time_us":1428599}]]}
I20260709 12:19:39.145220 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.151820 18266 leader_election.cc:304] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.138461 18194 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 10 to 8, term changed from 16 to 13, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65), VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66), VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67), VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 13 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.147073 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:39.153240 25704 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:39.154033 25704 raft_consensus.cc:2749] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:39.154115 18194 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 13 to 14, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 14 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.150707 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:39.159740 25675 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.037945 25609 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.163110 25609 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.165259 25609 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:39.172122 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:39.156412 18194 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.173233 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:39.177899 18371 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:39.179761 18194 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 13 to 15, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.181751 18194 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 15 to 18, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:39.183806 18194 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.185847 18194 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 12 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:39.189414 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 213610us
I20260709 12:19:39.191550 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.194989 19038 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:19:39.196867 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 233148us
W20260709 12:19:39.204942 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 257176us
I20260709 12:19:39.199254 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.204972 18194 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.206573 18266 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.211493 18293 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:39.201565 18371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:39.218549 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:39.219273 18446 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 24: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:39.220028 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.220605 19035 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:39.217229 19006 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.227088 18194 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 11 to 14, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 14 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:39.227643 25785 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:39.236783 19006 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.228546 18194 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 11 to 13, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 13 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:39.208401 25565 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:39.208907 25684 raft_consensus.cc:2749] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:39.238312 25785 raft_consensus.cc:2749] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:39.245862 25453 consensus_queue.cc:1048] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:39.248725 25686 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:39.249810 25702 consensus_queue.cc:1059] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:39.249996 19038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:39.292002 19038 raft_consensus.cc:606] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:39.292445 19038 raft_consensus.cc:614] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:39.251539 25616 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:39.255028 18194 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:39.260437 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
W20260709 12:19:39.281157 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 53351.1us
I20260709 12:19:39.283679 19006 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:39.294283 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 48662.5us
I20260709 12:19:39.212687 25503 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:19:39.325135 19038 raft_consensus.cc:1240] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:39.326411 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80bc05a330c64471aaa556af75bea031" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.328572 19035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67a1783f49244bd5ba98721205d1ff6e" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.331192 19043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea246fa8cc7e4edbb136da2ee92fb1d3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:39.331943 19043 raft_consensus.cc:2393] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 18: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:39.320528 25726 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:39.350356 25702 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:39.299376 25411 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 12:19:39.352164 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 91805.6us
I20260709 12:19:39.362454 18194 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 15 to 16, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.349293 25786 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:39.355731 25702 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:39.375512 25702 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:39.365299 25709 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.379583 25709 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.381706 25709 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.383536 25786 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.389143 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:39.393465 25793 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.002s
W20260709 12:19:39.396332 25794 log.cc:927] Time spent T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:19:39.411803 25620 raft_consensus.cc:3055] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:39.412266 25620 raft_consensus.cc:740] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:39.413164 25620 consensus_queue.cc:260] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:39.414445 25620 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:39.436507 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.437194 19039 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 19.
I20260709 12:19:39.439572 18293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98a95c8e3c14bcfa51b054f4829293b"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:39.440160 18293 raft_consensus.cc:606] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:39.440479 18293 raft_consensus.cc:614] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:39.445534 25590 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:39.445974 25590 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:39.446888 25590 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.448230 25590 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.461573 18263 leader_election.cc:304] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:39.472606 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a971a418238d45d288b49960c0abeea0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:39.473413 18293 raft_consensus.cc:3060] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:39.495028 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:39.422979 25702 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.500643 18295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:39.510740 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:39.518695 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:39.520824 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:39.521888 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:39.542912 18293 raft_consensus.cc:2468] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:39.520007 25481 consensus_queue.cc:1059] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:19:39.562295 18375 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:39.585413 18294 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:39.635640 19043 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:39.646633 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:39.615659 25683 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.676349 19041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.650396 25786 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.646055 25787 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:19:39.716249 18263 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:39.744939 19006 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.749401 25725 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:39.750857 19006 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.752064 25702 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.763868 25778 raft_consensus.cc:2749] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:39.765674 25627 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:39.767935 19006 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.768546 25711 raft_consensus.cc:3055] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:39.771435 25711 raft_consensus.cc:740] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:39.772246 25711 consensus_queue.cc:260] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:39.773418 25711 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:39.784400 19006 leader_election.cc:304] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:39.789048 25586 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:39.796684 25467 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:39.798619 25411 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.805905 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
W20260709 12:19:39.845213 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 174579us
I20260709 12:19:39.675747 25683 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:39.864953 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 52662.2us
I20260709 12:19:39.842223 25788 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:39.850993 25777 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:39.873534 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 89213.2us
I20260709 12:19:39.879799 25606 raft_consensus.cc:2804] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:39.883620 18446 raft_consensus.cc:2393] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3da7ad8fa20a4591963b660e4c848e7f in current term 24: Already voted for candidate 177254018ba7448aa2b9110c9c9be9e3 in this term.
I20260709 12:19:39.894369 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:39.895193 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 24.
I20260709 12:19:39.884673 25683 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:39.799130 25411 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:39.902558 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 226057us
I20260709 12:19:39.896951 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:39.906075 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
W20260709 12:19:39.905552 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 187202us
I20260709 12:19:39.903236 25606 raft_consensus.cc:697] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:39.938210 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:39.939603 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:39.946027 18375 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 24.
I20260709 12:19:39.914682 25806 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.914278 25707 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:19:39.949194 25796 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:39.968953 19037 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:39.972026 18338 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:39.951893 25707 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.982738 25707 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:39.953401 25806 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:40.023473 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 46609.7us
I20260709 12:19:39.996580 25809 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:40.051018 25777 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:40.051574 25777 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.039539 25606 consensus_queue.cc:237] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.060969 25806 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:40.048202 25805 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:19:40.098244 18291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:40.101691 18291 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 16.
I20260709 12:19:40.046060 25809 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:40.107890 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 113334us
I20260709 12:19:40.132568 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:40.133445 18293 raft_consensus.cc:2410] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:40.111260 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:40.150197 25411 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:40.150345 25797 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:40.150871 25797 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.151264 25797 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.115187 25707 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.152456 25777 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.123521 25809 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:40.164580 25815 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:19:40.168874 25797 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:40.154686 25803 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.363s	user 0.005s	sys 0.000s
I20260709 12:19:40.182739 25777 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:40.183809 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:40.186705 25707 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:40.187547 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:40.188205 18299 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.173825 25797 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:40.196601 25822 log.cc:927] Time spent T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:40.200065 18299 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:40.202266 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:40.201609 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:40.207603 25824 log.cc:927] Time spent T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
W20260709 12:19:40.216272 25826 log.cc:927] Time spent T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:19:40.216358 18338 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:40.226660 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:40.229142 18446 raft_consensus.cc:3060] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.230126 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:40.237033 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83652a40670f4224a56a5a1347b9d1a1" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:40.240957 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:40.243314 25816 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.244786 18375 raft_consensus.cc:2393] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 3da7ad8fa20a4591963b660e4c848e7f in this term.
I20260709 12:19:40.245210 25620 raft_consensus.cc:2804] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:40.245744 25620 raft_consensus.cc:697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:40.246589 25620 consensus_queue.cc:237] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:40.251008 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ef0235d410491ab47198988075f6e4" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:40.251715 19040 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:40.261425 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:40.263654 18446 raft_consensus.cc:2468] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
W20260709 12:19:40.267890 25827 log.cc:927] Time spent T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:19:40.268335 25686 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.287562 19040 raft_consensus.cc:2468] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 25.
I20260709 12:19:40.313011 18412 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:40.315390 25550 raft_consensus.cc:2804] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:19:40.316119 25550 raft_consensus.cc:697] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:40.317268 25550 consensus_queue.cc:237] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.347999 18299 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:40.431346 25820 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:19:40.452697 25798 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:40.453121 25798 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:40.453884 25798 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.455214 25798 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.457387 18443 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:40.470242 25616 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:40.528287 18293 raft_consensus.cc:1275] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:40.530043 25608 consensus_queue.cc:1048] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:40.572865 18195 catalog_manager.cc:5697] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.574661 18195 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 19, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 19 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.576051 18195 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.577421 18195 catalog_manager.cc:5697] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 17, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 17 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.578660 18195 catalog_manager.cc:5697] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 20, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 20 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
W20260709 12:19:40.580869 18194 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 296) took 32680 ms (client timeout 15000 ms). Trace:
W20260709 12:19:40.581972 18194 rpcz_store.cc:269] 0709 12:19:07.900693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.900942 (+   249us) service_pool.cc:224] Handling call
0709 12:19:08.394240 (+493298us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b5400002490. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d58640 00007f3c6ec2f846 00007f3c6ed3990e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:40.580785 (+32186545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:39.533840 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:39.671541 (+137701us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:39.673927 (+  2386us) write_op.cc:435] Acquired schema lock
0709 12:19:39.675850 (+  1923us) tablet.cc:662] Decoding operations
0709 12:19:39.777359 (+101509us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:39.777440 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:39.777467 (+    27us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:19:40.008050 (+230583us) tablet.cc:701] Row locks acquired
0709 12:19:40.009905 (+  1855us) write_op.cc:260] PREPARE: finished
0709 12:19:40.037854 (+ 27949us) write_op.cc:270] Start()
0709 12:19:40.040850 (+  2996us) write_op.cc:276] Timestamp: P: 1783599580023961 usec, L: 0
0709 12:19:40.042489 (+  1639us) op_driver.cc:348] REPLICATION: starting
0709 12:19:40.055453 (+ 12964us) log.cc:844] Serialized 5080 byte log entry
0709 12:19:40.200828 (+145375us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b4400022b4c. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65cb5c85 00007f3c65cb66a2 00007f3c68a281db 00007f3c68a23aaf 00007f3c68a1786e 00007f3c68a16bb6 00007f3c68a167d8 00007f3c68a20221 00007f3c68a201d9 00007f3c68a20169
0709 12:19:40.380736 (+179908us) op_driver.cc:464] REPLICATION: finished
0709 12:19:40.388002 (+  7266us) write_op.cc:301] APPLY: starting
0709 12:19:40.388087 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:40.394578 (+  6491us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:40.394601 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:40.483002 (+ 88401us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:40.483267 (+   265us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:19:40.483312 (+    45us) write_op.cc:312] APPLY: finished
0709 12:19:40.483602 (+   290us) log.cc:844] Serialized 113 byte log entry
0709 12:19:40.483765 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:40.483910 (+   145us) write_op.cc:454] Released schema lock
0709 12:19:40.483958 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:40.535821 (+ 51863us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:40.535857 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":142096,"reported_tablets":47,"spinlock_wait_cycles":220537088,"child_traces":[["op",{"apply.queue_time_us":7016,"apply.run_cpu_time_us":15616,"apply.run_wall_time_us":147808,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54239,"num_ops":12,"prepare.queue_time_us":85146,"prepare.run_cpu_time_us":23538,"prepare.run_wall_time_us":577121,"raft.queue_time_us":171782,"raft.run_cpu_time_us":4681,"raft.run_wall_time_us":7347,"replication_time_us":338089,"spinlock_wait_cycles":9945344,"thread_start_us":225821,"threads_started":4,"wal-append.queue_time_us":73009}]]}
I20260709 12:19:40.580163 18195 catalog_manager.cc:5697] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 21, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 21 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.587397 18195 catalog_manager.cc:5697] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 22, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 22 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.588896 18195 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.590397 18195 catalog_manager.cc:5697] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 18, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 18 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.593204 18195 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 21, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 21 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.595013 18195 catalog_manager.cc:5697] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 19, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 19 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.596532 18195 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 8 to 10, term changed from 13 to 18, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to <none>, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65), VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66), VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67), VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 18 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.599740 18195 catalog_manager.cc:5697] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 24, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 24 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.601210 18195 catalog_manager.cc:5697] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 19, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 19 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.602495 18195 catalog_manager.cc:5697] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 14 to 21, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 21 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.603967 18195 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 16, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 16 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.605502 18195 catalog_manager.cc:5697] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 23, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 23 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.607009 18195 catalog_manager.cc:5697] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 20 to 24, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 24 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.608295 18195 catalog_manager.cc:5697] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 20 to 22, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.610442 18195 catalog_manager.cc:5697] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.611665 18195 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.612982 18195 catalog_manager.cc:5697] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.614418 18195 catalog_manager.cc:5697] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.615758 18195 catalog_manager.cc:5697] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 22 to 23, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 23 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.617179 18195 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.618448 18195 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:40.619632 18195 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.620764 18195 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 15 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:40.647995 18443 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.649691 25620 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:40.664948 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:40.702594 18299 raft_consensus.cc:1275] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.704833 19037 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 24. Current term is 25. Ops: []
I20260709 12:19:40.708459 25809 consensus_queue.cc:1048] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:19:40.708070 25620 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:19:40.728013 25809 raft_consensus.cc:3055] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:19:40.753269 18446 raft_consensus.cc:1240] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 LEADER]: Rejecting Update request from peer 3da7ad8fa20a4591963b660e4c848e7f for earlier term 24. Current term is 25. Ops: []
I20260709 12:19:40.740517 25809 raft_consensus.cc:740] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:40.758915 25810 consensus_queue.cc:1059] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:40.759677 25810 consensus_queue.cc:1243] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Peer 177254018ba7448aa2b9110c9c9be9e3 log is divergent from this leader: its last log entry 25.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:19:40.767395 25809 consensus_queue.cc:260] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.771834 25809 raft_consensus.cc:3060] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:40.762918 25411 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:40.772818 25411 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.777733 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:40.778339 18375 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:40.779328 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:40.782102 25493 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:40.783656 25493 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:40.784051 25493 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.788390 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:40.789119 19040 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:40.790351 25411 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:40.793289 18296 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:40.793932 18296 raft_consensus.cc:606] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:40.805220 18299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:40.819294 18299 raft_consensus.cc:606] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:40.828855 25493 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.831593 25493 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:40.832549 18373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:40.832707 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:40.855528 19040 raft_consensus.cc:2393] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 17: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:40.844951 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:40.856020 19042 raft_consensus.cc:606] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:40.856406 19042 raft_consensus.cc:614] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.857718 18373 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:40.860939 19042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:40.861515 19042 raft_consensus.cc:606] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:40.861737 18415 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:40.862709 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:19:40.862869 25493 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:40.863611 25493 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:40.864773 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:40.864421 25493 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:40.865684 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:40.886294 25778 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.886852 25778 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:40.888921 25778 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:40.891283 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:40.904136 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:40.904778 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:40.905853 18266 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:40.916392 25550 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:40.916961 25550 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:40.921718 25774 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:40.922194 25774 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.922567 25774 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.937520 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:40.974850 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:40.975551 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:40.976768 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
W20260709 12:19:41.004010 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 100721us
I20260709 12:19:41.004953 18195 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 299) took 1311 ms. Trace:
I20260709 12:19:41.005362 18195 rpcz_store.cc:276] 0709 12:19:39.670471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:39.670632 (+   161us) service_pool.cc:224] Handling call
0709 12:19:39.851859 (+181227us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b5400002490. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c65c17db1 00007f3c65d57b71 00007f3c65d580b2 00007f3c6ecd1759 00007f3c6ec69a76 00007f3c6ec3ad58 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc
0709 12:19:40.981630 (+1129771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:40.646800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:40.647355 (+   555us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:40.647389 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:19:40.647407 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:40.649257 (+  1850us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:40.649338 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:40.649362 (+    24us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:19:40.651614 (+  2252us) tablet.cc:701] Row locks acquired
0709 12:19:40.651633 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:19:40.651765 (+   132us) write_op.cc:270] Start()
0709 12:19:40.651883 (+   118us) write_op.cc:276] Timestamp: P: 1783599580651719 usec, L: 0
0709 12:19:40.651906 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:19:40.652616 (+   710us) log.cc:844] Serialized 10958 byte log entry
0709 12:19:40.656756 (+  4140us) op_driver.cc:464] REPLICATION: finished
0709 12:19:40.659405 (+  2649us) write_op.cc:301] APPLY: starting
0709 12:19:40.659536 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:40.773436 (+113900us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:40.773539 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:40.892080 (+118541us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:40.892429 (+   349us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 12:19:40.892473 (+    44us) write_op.cc:312] APPLY: finished
0709 12:19:40.895639 (+  3166us) log.cc:844] Serialized 244 byte log entry
0709 12:19:40.895806 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:40.896040 (+   234us) write_op.cc:454] Released schema lock
0709 12:19:40.896104 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:40.924269 (+ 28165us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:40.924314 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15934,"reported_tablets":36,"spinlock_wait_cycles":6944384,"child_traces":[["op",{"apply.queue_time_us":2420,"apply.run_cpu_time_us":34898,"apply.run_wall_time_us":254283,"cfile_cache_hit":112,"cfile_cache_hit_bytes":116928,"num_ops":28,"prepare.queue_time_us":17474,"prepare.run_cpu_time_us":6089,"prepare.run_wall_time_us":9582,"replication_time_us":4695,"thread_start_us":17145,"threads_started":1}]]}
W20260709 12:19:41.031613 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 65678.7us
I20260709 12:19:40.974035 25550 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:41.046140 25798 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:41.083681 25798 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:41.085223 25798 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:41.089938 18197 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 10 to -1, term changed from 18 to 6, leader changed from <none> to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67), VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) evicted. New cstate: current_term: 6 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:41.107498 25846 raft_consensus.cc:993] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:41.108067 25846 raft_consensus.cc:1081] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
W20260709 12:19:41.119894 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 143189us
I20260709 12:19:41.123678 18375 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:19:41.124150 18375 pending_rounds.cc:85] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting all ops after (but not including) 15
I20260709 12:19:41.124516 18375 pending_rounds.cc:107] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:19:41.124882 18375 raft_consensus.cc:2889] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:41.131898 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 158367us
W20260709 12:19:41.133078 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 159957us
I20260709 12:19:41.138183 19040 raft_consensus.cc:1275] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:19:41.137704 25616 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.153239 25616 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.156387 25616 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:41.165268 25798 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:41.167846 25798 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:41.168538 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:41.169736 25774 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:41.180653 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:41.186326 18474 heartbeater.cc:646] Failed to heartbeat to 127.16.193.126:46027 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.16.193.126:46027 timed out after 15.000s (SENT)
I20260709 12:19:41.186700 25411 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:41.196514 18375 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:41.196952 18373 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:19:41.198886 18373 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:41.199285 18373 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:41.201792 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:41.216507 18296 raft_consensus.cc:2368] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21.
I20260709 12:19:41.217334 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:41.210093 18373 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.254400 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:41.283914 18373 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:41.242187 25493 consensus_queue.cc:1048] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:19:41.247700 25843 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:41.320446 25843 raft_consensus.cc:493] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.320957 25843 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.324767 25774 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:41.347162 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 139922us
I20260709 12:19:41.351670 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:41.355213 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:41.355855 19042 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:19:41.367704 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 164890us
I20260709 12:19:41.369442 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.370409 18444 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.387719 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:41.388479 18374 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.406016 18444 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
W20260709 12:19:41.398355 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 180835us
I20260709 12:19:41.412051 18371 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:41.414709 25586 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:41.417927 25586 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.419019 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4622d18b474b659ebea19e9aad0d05" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:41.419660 18293 raft_consensus.cc:3055] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:41.419974 18293 raft_consensus.cc:740] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:41.420255 25586 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:41.420719 18293 consensus_queue.cc:260] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.421922 18293 raft_consensus.cc:3060] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:41.426029 25863 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.426607 25863 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.431063 25863 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:41.434762 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:41.392253 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 190527us
I20260709 12:19:41.438414 19040 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 in current term 18: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:41.437419 19042 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:41.444854 18293 raft_consensus.cc:2468] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 24.
I20260709 12:19:41.446902 18374 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:41.442939 18338 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:41.456712 25620 raft_consensus.cc:2804] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:41.465488 25620 raft_consensus.cc:697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:41.466600 25620 consensus_queue.cc:237] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.476279 25852 raft_consensus.cc:993] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:41.476913 25852 raft_consensus.cc:1081] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:41.487495 18369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:56140
I20260709 12:19:41.488224 18369 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:41.487793 25843 raft_consensus.cc:515] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.488653 18369 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.489109 19037 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:41.507601 18264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 89135.8us
I20260709 12:19:41.508332 18264 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:41.525563 25863 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:41.526093 25863 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:41.526863 25863 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
W20260709 12:19:41.532653 18265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.545624 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:41.546447 18446 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 17.
I20260709 12:19:41.555456 18264 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:41.553472 18369 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.565387 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.566224 18444 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.575614 18369 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:41.529424 25493 consensus_queue.cc:1059] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:19:41.541452 25851 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.590021 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:41.591929 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 162175us
I20260709 12:19:41.594250 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:41.601676 18444 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:41.603922 18339 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:41.590657 25857 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:41.613770 25854 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:19:41.611883 25620 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:41.655251 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:41.656078 18369 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.613648 25707 raft_consensus.cc:2764] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:19:41.597505 25851 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.662122 25851 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:41.662743 19037 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 18.
I20260709 12:19:41.665472 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:41.666163 18444 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 17.
I20260709 12:19:41.613539 25857 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.667717 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:41.675907 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 137776us
I20260709 12:19:41.679978 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:41.680570 19037 raft_consensus.cc:3055] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:41.680843 19037 raft_consensus.cc:740] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:41.681618 19037 consensus_queue.cc:260] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.682826 19037 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.669049 25620 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:41.687704 25620 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.689352 25870 raft_consensus.cc:993] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:41.689821 25870 raft_consensus.cc:1081] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
W20260709 12:19:41.697731 25868 log.cc:927] Time spent T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:19:41.672672 25857 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.708438 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:41.712352 25550 raft_consensus.cc:3055] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:41.712764 25550 raft_consensus.cc:740] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:41.713541 25550 consensus_queue.cc:260] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.714648 25550 raft_consensus.cc:3060] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.715629 25843 leader_election.cc:290] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:41.696692 25616 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:41.716941 25616 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.717280 25616 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.719841 18412 leader_election.cc:400] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 20. Current term is 21.
I20260709 12:19:41.723421 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5398d6474dd4eed946f45af9214825c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.728359 18369 raft_consensus.cc:2468] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 18.
I20260709 12:19:41.746896 18374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:19:41.769817 18374 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:41.770280 18374 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.767361 25616 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.779143 18374 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:41.811389 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.811991 18441 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.820498 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:41.783924 19009 leader_election.cc:304] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:41.822693 25608 raft_consensus.cc:2804] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:41.823189 25608 raft_consensus.cc:697] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:41.824050 25608 consensus_queue.cc:237] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.785809 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.828464 18447 raft_consensus.cc:3060] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.834062 19037 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:41.826905 18441 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:41.782991 18375 raft_consensus.cc:1240] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer 177254018ba7448aa2b9110c9c9be9e3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:41.804214 18444 raft_consensus.cc:2410] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:19:41.806905 25857 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.821341 18296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:41.837800 18339 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:41.780493 25616 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:41.842876 18374 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:41.852522 18299 raft_consensus.cc:2368] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:41.857800 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c08deb8e86480a878b17c55d6bc089" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:41.858359 18294 raft_consensus.cc:3055] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:41.858644 18294 raft_consensus.cc:740] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:41.859336 18294 consensus_queue.cc:260] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:41.860409 18294 raft_consensus.cc:3060] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.867679 25857 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:41.874097 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:41.845602 25874 raft_consensus.cc:2804] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:41.875936 25874 raft_consensus.cc:697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:41.878957 19042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:41.881466 19042 raft_consensus.cc:2393] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e23c0ce8d89940da8eac6a6002e349c5 in current term 19: Already voted for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in this term.
I20260709 12:19:41.885630 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:41.886672 25863 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:41.876822 25874 consensus_queue.cc:237] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:41.892031 18294 raft_consensus.cc:2468] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:41.898870 18447 raft_consensus.cc:2468] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:41.899972 19007 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:41.901216 18412 leader_election.cc:400] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:41.901572 18412 leader_election.cc:403] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:41.873613 25616 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:41.902691 25608 raft_consensus.cc:2804] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:41.904562 25608 raft_consensus.cc:697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:41.905462 25608 consensus_queue.cc:237] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.914151 25849 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771) with higher term. Message: Invalid argument: T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 177254018ba7448aa2b9110c9c9be9e3 for earlier term 18. Current term is 19.
I20260709 12:19:41.917237 25616 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:41.928023 18447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:41.928920 18447 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:41.930119 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:41.930997 25850 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:41.931421 25850 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:41.931615 18446 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:41.931802 25850 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:41.934762 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5" is_pre_election: true
I20260709 12:19:41.935536 18294 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:41.936275 25857 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:41.927037 25616 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:42.008272 25850 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.032368 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:42.034044 18195 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 10, term changed from 6 to 18, VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:42.048743 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 96913.2us
I20260709 12:19:42.054016 18294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:42.054767 18294 raft_consensus.cc:2393] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 991dcd2a70c74fdb89ca9494cc09fb7b in current term 20: Already voted for candidate e23c0ce8d89940da8eac6a6002e349c5 in this term.
I20260709 12:19:42.054924 18299 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:19:42.075654 25850 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:42.076627 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:42.091310 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:42.092353 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:42.098225 18444 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:42.104375 25850 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:42.106243 25850 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:42.106441 25620 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:42.107185 25850 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.124596 18195 catalog_manager.cc:5697] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 11 to 17, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 17 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:42.138873 25874 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 12:19:42.178586 19042 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:42.187556 18373 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:42.189764 19037 raft_consensus.cc:1275] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:19:42.195387 25860 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:42.194371 25889 consensus_queue.cc:1048] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:19:42.192160 25809 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:19:42.356369 18293 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:19:42.389719 18197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 293) took 103473 ms (client timeout 15000 ms). Trace:
I20260709 12:19:42.394145 25843 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:42.403126 18444 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:19:42.403609 18444 pending_rounds.cc:85] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Aborting all ops after (but not including) 12
I20260709 12:19:42.403949 18444 pending_rounds.cc:107] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:19:42.404261 18444 raft_consensus.cc:2889] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:42.437598 25779 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.438607 25779 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.443419 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:42.444497 18444 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 20.
I20260709 12:19:42.446233 18441 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:42.448377 25620 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:42.456888 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 
I20260709 12:19:42.459715 25860 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:42.460156 25860 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:42.460745 25860 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:42.462641 18195 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 300) took 1416 ms. Trace:
I20260709 12:19:42.473762 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:42.432125 25843 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:19:42.442037 25779 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:42.390391 18197 rpcz_store.cc:269] 0709 12:17:58.915999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.012920 (+ 96921us) service_pool.cc:224] Handling call
0709 12:18:00.892638 (+1879718us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:42.389437 (+101496799us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:41.403857 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:41.482876 (+ 79019us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:41.482914 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:19:41.482935 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:41.529274 (+ 46339us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:41.530621 (+  1347us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:41.542500 (+ 11879us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:41.552163 (+  9663us) tablet.cc:701] Row locks acquired
0709 12:19:41.552183 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:19:41.552317 (+   134us) write_op.cc:270] Start()
0709 12:19:41.552433 (+   116us) write_op.cc:276] Timestamp: P: 1783599581552276 usec, L: 0
0709 12:19:41.552456 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:19:41.553940 (+  1484us) log.cc:844] Serialized 541 byte log entry
0709 12:19:41.879268 (+325328us) op_driver.cc:464] REPLICATION: finished
0709 12:19:41.924317 (+ 45049us) write_op.cc:301] APPLY: starting
0709 12:19:41.924469 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:41.925680 (+  1211us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:41.925765 (+    85us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:41.938028 (+ 12263us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:41.952646 (+ 14618us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:19:41.956492 (+  3846us) write_op.cc:312] APPLY: finished
0709 12:19:41.977610 (+ 21118us) log.cc:844] Serialized 25 byte log entry
0709 12:19:41.995708 (+ 18098us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:41.996757 (+  1049us) write_op.cc:454] Released schema lock
0709 12:19:41.998609 (+  1852us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:42.006571 (+  7962us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:42.006599 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":981,"mutex_wait_us":328792,"reported_tablets":42,"spinlock_wait_cycles":960025984,"child_traces":[["op",{"apply.queue_time_us":44711,"apply.run_cpu_time_us":3822,"apply.run_wall_time_us":82283,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8303,"num_ops":1,"prepare.queue_time_us":191133,"prepare.run_cpu_time_us":3333,"prepare.run_wall_time_us":158445,"replication_time_us":326625,"spinlock_wait_cycles":1408,"thread_start_us":88608,"threads_started":2}]]}
I20260709 12:19:42.498742 18294 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:42.463080 18195 rpcz_store.cc:276] 0709 12:19:41.045950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.046093 (+   143us) service_pool.cc:224] Handling call
0709 12:19:42.462422 (+1416329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:42.156851 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:42.214636 (+ 57785us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:42.215122 (+   486us) write_op.cc:435] Acquired schema lock
0709 12:19:42.216485 (+  1363us) tablet.cc:662] Decoding operations
0709 12:19:42.217020 (+   535us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:42.217118 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:42.217151 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:19:42.217505 (+   354us) tablet.cc:701] Row locks acquired
0709 12:19:42.217528 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:19:42.217716 (+   188us) write_op.cc:270] Start()
0709 12:19:42.217843 (+   127us) write_op.cc:276] Timestamp: P: 1783599582217666 usec, L: 0
0709 12:19:42.217876 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:19:42.218641 (+   765us) log.cc:844] Serialized 987 byte log entry
0709 12:19:42.370143 (+151502us) op_driver.cc:464] REPLICATION: finished
0709 12:19:42.371388 (+  1245us) write_op.cc:301] APPLY: starting
0709 12:19:42.371566 (+   178us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:42.374106 (+  2540us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:42.374278 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:42.406019 (+ 31741us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:42.406613 (+   594us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:19:42.406684 (+    71us) write_op.cc:312] APPLY: finished
0709 12:19:42.407115 (+   431us) log.cc:844] Serialized 33 byte log entry
0709 12:19:42.407330 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:42.407547 (+   217us) write_op.cc:454] Released schema lock
0709 12:19:42.407606 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:42.416209 (+  8603us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:42.419074 (+  2865us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":95507,"reported_tablets":7,"spinlock_wait_cycles":91392,"child_traces":[["op",{"apply.queue_time_us":775,"apply.run_cpu_time_us":6403,"apply.run_wall_time_us":41744,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8352,"num_ops":2,"prepare.queue_time_us":28878,"prepare.run_cpu_time_us":3294,"prepare.run_wall_time_us":62105,"replication_time_us":151884,"thread_start_us":35485,"threads_started":1}]]}
I20260709 12:19:42.507547 18444 raft_consensus.cc:1275] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:42.522792 18294 raft_consensus.cc:606] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:42.533955 18373 raft_consensus.cc:1275] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:42.545276 25608 consensus_queue.cc:1048] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:42.555615 25606 consensus_queue.cc:1048] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:42.618484 18294 raft_consensus.cc:1275] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:42.581198 25860 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.635607 25860 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Requested vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:42.638749 18443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:42.639539 18443 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:42.644990 18299 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:42.649205 19037 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:42.654764 25874 consensus_queue.cc:1048] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:42.680145 25779 raft_consensus.cc:993] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:42.680691 25779 raft_consensus.cc:1081] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:42.713124 18443 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e23c0ce8d89940da8eac6a6002e349c5 in term 21.
I20260709 12:19:42.714030 18446 raft_consensus.cc:1240] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:42.715206 25867 consensus_queue.cc:1059] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:42.716423 25616 raft_consensus.cc:3055] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:42.716786 25616 raft_consensus.cc:740] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:42.714102 18369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:19:42.721088 18369 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:42.731880 18369 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:42.717741 25616 consensus_queue.cc:260] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.751838 25616 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:42.763808 19034 tablet_service.cc:1558] Processing DeleteTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with delete_type TABLET_DATA_TOMBSTONED (TS 991dcd2a70c74fdb89ca9494cc09fb7b not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:58728
I20260709 12:19:42.727141 18191 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 13 to 18, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 18 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:42.813684 18264 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:42.822172 25779 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:42.822736 25779 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Becoming Leader. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:42.823874 25779 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:42.854722 18191 catalog_manager.cc:5697] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 12 to 17, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:42.885458 18191 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 14 to 15, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:42.904667 18369 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:42.913031 25911 tablet_replica.cc:333] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:19:42.914335 25911 raft_consensus.cc:2243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:42.915427 25911 raft_consensus.cc:2272] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:42.919222 25911 ts_tablet_manager.cc:1916] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:42.929314 18444 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:34614
I20260709 12:19:42.929961 18444 raft_consensus.cc:606] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:42.936430 18369 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:42.936625 18293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:42.937233 18293 raft_consensus.cc:3055] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:42.936002 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85f32970b7374dcf9b9bac47791c71e3" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:42.938074 18442 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:42.937536 18293 raft_consensus.cc:740] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:42.943580 18293 consensus_queue.cc:260] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:42.944773 18293 raft_consensus.cc:3060] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:42.955317 18442 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:42.960260 18339 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:42.960455 18293 raft_consensus.cc:2468] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
I20260709 12:19:42.938529 18444 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2ee63a449648bfb064fadee177cadb"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
mode: GRACEFUL
new_leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
 from {username='slave'} at 127.0.0.1:34614
I20260709 12:19:42.961139 25874 raft_consensus.cc:2804] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:42.961843 25874 raft_consensus.cc:697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:42.962862 25874 consensus_queue.cc:237] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:42.964960 18444 raft_consensus.cc:606] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Received request to transfer leadership to TS e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:42.965394 18444 raft_consensus.cc:614] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:42.968245 19039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:42.968961 19039 raft_consensus.cc:606] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:42.969267 19039 raft_consensus.cc:614] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:42.971324 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:19:42.973325 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:42.974484 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:42.991884 25914 log.cc:927] Time spent T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:19:43.015343 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 140941us
W20260709 12:19:43.345991 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44152.4us
I20260709 12:19:43.369516 25853 raft_consensus.cc:993] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:43.370071 25853 raft_consensus.cc:1081] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:43.408212 18374 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:46418
I20260709 12:19:43.408895 18374 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:43.409258 18374 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.439960 18374 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:43.460657 18412 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:43.462091 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:43.462723 18441 raft_consensus.cc:3055] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.463028 18441 raft_consensus.cc:740] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:43.463945 18441 consensus_queue.cc:260] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:43.465116 18441 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.472724 25911 ts_tablet_manager.cc:1929] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 12:19:43.473294 25911 log.cc:1199] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-3-root/wals/ea246fa8cc7e4edbb136da2ee92fb1d3
I20260709 12:19:43.473286 18374 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:43.480134 19039 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 12:19:43.480485 19039 pending_rounds.cc:85] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting all ops after (but not including) 16
I20260709 12:19:43.480729 19039 pending_rounds.cc:107] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 12:19:43.480949 19039 raft_consensus.cc:2889] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:43.482596 18441 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:43.488209 18182 catalog_manager.cc:5028] TS 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:19:43.488786 18339 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:43.490165 25874 raft_consensus.cc:2804] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:43.490697 25874 raft_consensus.cc:697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:43.494601 25874 consensus_queue.cc:237] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:43.508253 25923 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:43.513274 18299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977c0af9c8e94119b3d18a0ee7401458" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:43.513890 18299 raft_consensus.cc:3060] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.532143 18299 raft_consensus.cc:2468] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 17.
I20260709 12:19:43.536015 18441 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:43.537744 25889 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:43.578505 18444 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Refusing update from remote peer e23c0ce8d89940da8eac6a6002e349c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:19:43.581992 25856 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:43.603736 18197 catalog_manager.cc:5697] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 9 to 25. New cstate: current_term: 25 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.605405 18197 catalog_manager.cc:5697] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 18, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 18 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.608471 18197 catalog_manager.cc:5697] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 23 to 24, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 24 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:43.619485 18299 raft_consensus.cc:1275] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:43.609916 18197 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 19, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:43.663977 18197 catalog_manager.cc:5697] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 20 to 21, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 21 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:43.666103 18197 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 21, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 21 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } }
I20260709 12:19:43.732703 18197 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 301) took 1045 ms. Trace:
I20260709 12:19:43.733197 18197 rpcz_store.cc:276] 0709 12:19:42.687849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.688043 (+   194us) service_pool.cc:224] Handling call
0709 12:19:43.732616 (+1044573us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:43.679840 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:43.680365 (+   525us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:43.680399 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:19:43.680418 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:43.681224 (+   806us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:43.681274 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:43.681300 (+    26us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:19:43.681834 (+   534us) tablet.cc:701] Row locks acquired
0709 12:19:43.681853 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:19:43.681991 (+   138us) write_op.cc:270] Start()
0709 12:19:43.682114 (+   123us) write_op.cc:276] Timestamp: P: 1783599583681945 usec, L: 0
0709 12:19:43.682133 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:19:43.682767 (+   634us) log.cc:844] Serialized 2458 byte log entry
0709 12:19:43.689780 (+  7013us) op_driver.cc:464] REPLICATION: finished
0709 12:19:43.690107 (+   327us) write_op.cc:301] APPLY: starting
0709 12:19:43.690186 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:43.694689 (+  4503us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:43.694800 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:43.699308 (+  4508us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:43.699520 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 12:19:43.699575 (+    55us) write_op.cc:312] APPLY: finished
0709 12:19:43.699816 (+   241us) log.cc:844] Serialized 65 byte log entry
0709 12:19:43.700633 (+   817us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.700749 (+   116us) write_op.cc:454] Released schema lock
0709 12:19:43.700798 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:43.701968 (+  1170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.702006 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":91837,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":9689,"apply.run_wall_time_us":11826,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25056,"num_ops":6,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":3235,"prepare.run_wall_time_us":9740,"replication_time_us":7515}]]}
I20260709 12:19:43.765790 18194 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.767817 18194 catalog_manager.cc:5697] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 18 to 19, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 19 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.769582 18194 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.774488 18194 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 17 to 19, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 19 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.776790 18194 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 13 to 21, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 21 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:43.803040 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 87435.9us
I20260709 12:19:43.803870 18194 catalog_manager.cc:5697] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 19 to 20, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 20 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:43.806288 18194 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 17 to 22, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 22 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.811281 25874 consensus_queue.cc:1048] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:43.811121 18194 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 11 to 15, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 15 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:43.847355 18194 catalog_manager.cc:5697] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 16 to 22, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:43.861390 18191 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 299) took 29938 ms (client timeout 15000 ms). Trace:
W20260709 12:19:43.864498 18191 rpcz_store.cc:269] 0709 12:19:13.922760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.923012 (+   252us) service_pool.cc:224] Handling call
0709 12:19:14.052771 (+129759us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:43.861288 (+29808517us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:43.264374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:43.290130 (+ 25756us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:43.290181 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:19:43.290200 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:43.290766 (+   566us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:43.290875 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:43.290904 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:19:43.291390 (+   486us) tablet.cc:701] Row locks acquired
0709 12:19:43.291438 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:19:43.291586 (+   148us) write_op.cc:270] Start()
0709 12:19:43.291744 (+   158us) write_op.cc:276] Timestamp: P: 1783599583291524 usec, L: 0
0709 12:19:43.291765 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:19:43.292385 (+   620us) log.cc:844] Serialized 1302 byte log entry
0709 12:19:43.447594 (+155209us) op_driver.cc:464] REPLICATION: finished
0709 12:19:43.536659 (+ 89065us) write_op.cc:301] APPLY: starting
0709 12:19:43.536758 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:43.541074 (+  4316us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:43.541152 (+    78us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:43.543110 (+  1958us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:43.543315 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:19:43.543358 (+    43us) write_op.cc:312] APPLY: finished
0709 12:19:43.551545 (+  8187us) log.cc:844] Serialized 41 byte log entry
0709 12:19:43.551741 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.551868 (+   127us) write_op.cc:454] Released schema lock
0709 12:19:43.551960 (+    92us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":1015254,"reported_tablets":47,"spinlock_wait_cycles":23831936,"child_traces":[["op",{"apply.queue_time_us":88809,"apply.run_cpu_time_us":5826,"apply.run_wall_time_us":15723,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16655,"num_ops":3,"prepare.queue_time_us":187436,"prepare.run_cpu_time_us":3378,"prepare.run_wall_time_us":41052,"replication_time_us":153166,"spinlock_wait_cycles":81664,"thread_start_us":232807,"threads_started":2}]]}
I20260709 12:19:43.941756 18197 catalog_manager.cc:5697] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 19 to 20, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 20 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:43.964246 18445 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:43.950495 18197 catalog_manager.cc:5697] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 16 to 17, leader changed from 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 17 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:43.968175 25924 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:44.052699 18299 raft_consensus.cc:1275] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:44.059661 25945 ts_tablet_manager.cc:933] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:44.064306 25942 consensus_queue.cc:1048] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:44.091810 25945 tablet_copy_client.cc:323] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:19:44.093955 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:19:44.094589 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet ea246fa8cc7e4edbb136da2ee92fb1d3 from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3
I20260709 12:19:44.157015 18190 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 15 to 18, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 18 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:44.167212 18194 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 298) took 2619 ms. Trace:
I20260709 12:19:44.166299 18459 tablet_copy_source_session.cc:215] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:44.171590 18194 rpcz_store.cc:276] 0709 12:19:41.548269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.548424 (+   155us) service_pool.cc:224] Handling call
0709 12:19:44.167138 (+2618714us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:43.855726 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:43.856463 (+   737us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:43.856504 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:19:43.856525 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:43.857458 (+   933us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:43.857512 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:43.857537 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:19:43.858316 (+   779us) tablet.cc:701] Row locks acquired
0709 12:19:43.858346 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:19:43.858500 (+   154us) write_op.cc:270] Start()
0709 12:19:43.858621 (+   121us) write_op.cc:276] Timestamp: P: 1783599583858451 usec, L: 0
0709 12:19:43.858646 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:19:43.859327 (+   681us) log.cc:844] Serialized 3601 byte log entry
0709 12:19:43.873076 (+ 13749us) op_driver.cc:464] REPLICATION: finished
0709 12:19:43.875466 (+  2390us) write_op.cc:301] APPLY: starting
0709 12:19:43.875561 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:43.880584 (+  5023us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:43.880683 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:43.897649 (+ 16966us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:43.897925 (+   276us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:19:43.897987 (+    62us) write_op.cc:312] APPLY: finished
0709 12:19:43.898345 (+   358us) log.cc:844] Serialized 89 byte log entry
0709 12:19:43.898542 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.898697 (+   155us) write_op.cc:454] Released schema lock
0709 12:19:43.898748 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:43.899123 (+   375us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:43.899156 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":123895,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":2133,"apply.run_cpu_time_us":12623,"apply.run_wall_time_us":23632,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37584,"num_ops":9,"prepare.queue_time_us":144,"prepare.run_cpu_time_us":3968,"prepare.run_wall_time_us":17204,"replication_time_us":14269}]]}
I20260709 12:19:44.201642 25945 ts_tablet_manager.cc:1916] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 12:19:44.206185 18441 consensus_queue.cc:237] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:44.247907 18412 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:44.253062 18375 raft_consensus.cc:1275] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:44.256903 25918 consensus_queue.cc:1048] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:44.363564 25918 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 10 to 14, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:44.380616 18183 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:44.379988 25945 ts_tablet_manager.cc:1929] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 18.13
I20260709 12:19:44.388640 25945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea246fa8cc7e4edbb136da2ee92fb1d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:44.365801 18375 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 10 to 14, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:44.482995 18414 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.638899 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:44.825579 25945 tablet_copy_client.cc:806] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:44.858223 25945 tablet_copy_client.cc:670] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
W20260709 12:19:44.931883 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 98974.3us
I20260709 12:19:45.092203 18190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46430 (request call id 300) took 10378 ms. Trace:
I20260709 12:19:45.126406 18191 catalog_manager.cc:5697] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 10 to 14, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:45.097344 18190 rpcz_store.cc:276] 0709 12:19:34.713570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.716342 (+  2772us) service_pool.cc:224] Handling call
0709 12:19:35.344505 (+628163us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b5400002438. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c6ec57ab1 00007f3c6ec3ae43 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6 00007f3c671b626f
0709 12:19:44.287797 (+8943292us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:44.287844 (+    47us) write_op.cc:454] Released schema lock
0709 12:19:45.092092 (+804248us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:44.254488 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:44.255128 (+   640us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:44.255165 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:19:44.255182 (+    17us) tablet.cc:662] Decoding operations
0709 12:19:44.255554 (+   372us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:44.255607 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:44.255636 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:44.255818 (+   182us) tablet.cc:701] Row locks acquired
0709 12:19:44.255830 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:19:44.255981 (+   151us) write_op.cc:270] Start()
0709 12:19:44.256100 (+   119us) write_op.cc:276] Timestamp: P: 1783599584255940 usec, L: 0
0709 12:19:44.256126 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:19:44.256717 (+   591us) log.cc:844] Serialized 536 byte log entry
0709 12:19:44.258555 (+  1838us) op_driver.cc:464] REPLICATION: finished
0709 12:19:44.259060 (+   505us) write_op.cc:301] APPLY: starting
0709 12:19:44.259139 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:44.259930 (+   791us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:44.260009 (+    79us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:44.260617 (+   608us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:44.260758 (+   141us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:19:44.260800 (+    42us) write_op.cc:312] APPLY: finished
0709 12:19:44.261001 (+   201us) log.cc:844] Serialized 25 byte log entry
0709 12:19:44.261471 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:44.261551 (+    80us) write_op.cc:454] Released schema lock
0709 12:19:44.261591 (+    40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":298020,"reported_tablets":47,"spinlock_wait_cycles":5445888,"child_traces":[["op",{"apply.queue_time_us":267,"apply.run_cpu_time_us":2501,"apply.run_wall_time_us":2791,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4176,"num_ops":1,"prepare.queue_time_us":6655,"prepare.run_cpu_time_us":2484,"prepare.run_wall_time_us":2485,"replication_time_us":2298}]]}
I20260709 12:19:45.155455 19039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d"
dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:39604
I20260709 12:19:45.156145 19039 raft_consensus.cc:606] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:45.160626 18292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:45.161331 18292 raft_consensus.cc:606] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Received request to transfer leadership to TS 177254018ba7448aa2b9110c9c9be9e3
I20260709 12:19:45.164969 18292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743"
dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
mode: GRACEFUL
new_leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:57914
I20260709 12:19:45.165632 18292 raft_consensus.cc:606] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Received request to transfer leadership to TS 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:45.165933 25950 raft_consensus.cc:993] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:45.166522 25952 raft_consensus.cc:1081] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:45.168854 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:45.174968 25978 raft_consensus.cc:993] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: : Instructing follower 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:45.175251 18441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:48170
I20260709 12:19:45.175571 25978 raft_consensus.cc:1081] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Signalling peer 177254018ba7448aa2b9110c9c9be9e3 to start an election
I20260709 12:19:45.175911 18441 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:45.197881 18441 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.205566 18441 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:45.208480 18441 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:45.209754 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:45.210440 18369 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.220724 25945 tablet_copy_client.cc:538] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:45.223579 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:45.224287 18292 raft_consensus.cc:3055] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:45.224629 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:19:45.234884 18193 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 297) took 19227 ms (client timeout 15000 ms). Trace:
W20260709 12:19:45.235426 18193 rpcz_store.cc:269] 0709 12:19:26.007511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.037250 (+ 29739us) service_pool.cc:224] Handling call
0709 12:19:45.234680 (+19197430us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":369925,"reported_tablets":47,"spinlock_wait_cycles":11520}
I20260709 12:19:45.239670 25945 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:19:45.224668 18292 raft_consensus.cc:740] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:45.242296 18292 consensus_queue.cc:260] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:45.248139 18369 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:45.249591 18292 raft_consensus.cc:3060] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.254376 18375 raft_consensus.cc:1240] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Rejecting Update request from peer e23c0ce8d89940da8eac6a6002e349c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.260526 18442 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d"
dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
 from {username='slave'} at 127.0.0.1:46956
I20260709 12:19:45.261070 18442 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:45.261436 18442 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.273062 25952 raft_consensus.cc:993] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: : Instructing follower 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:45.273660 25952 raft_consensus.cc:1081] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Signalling peer 3da7ad8fa20a4591963b660e4c848e7f to start an election
I20260709 12:19:45.288676 18367 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743"
dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
 from {username='slave'} at 127.0.0.1:55012
I20260709 12:19:45.289269 18367 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:45.289692 18367 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.268869 18442 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.301649 18441 raft_consensus.cc:1240] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Rejecting Update request from peer 991dcd2a70c74fdb89ca9494cc09fb7b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:45.305696 25955 consensus_queue.cc:1059] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:45.306720 25955 raft_consensus.cc:3055] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:45.307130 25955 raft_consensus.cc:740] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:45.308048 25955 consensus_queue.cc:260] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.309283 25955 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.312201 18415 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:45.313592 25919 raft_consensus.cc:2804] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:45.314241 25919 raft_consensus.cc:697] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:45.315354 25919 consensus_queue.cc:237] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:45.321004 18292 raft_consensus.cc:2468] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:45.327986 18442 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:45.321046 18367 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.329666 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:45.330776 18367 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:45.359648 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:45.361378 19040 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.359555 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5461f19f144642beb790dcf7f1515a2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:45.368950 18367 raft_consensus.cc:3060] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.370760 18292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd86332c91f4c8d8616b22b5daaf743" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "e23c0ce8d89940da8eac6a6002e349c5"
I20260709 12:19:45.371702 18292 raft_consensus.cc:3055] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:45.371991 18292 raft_consensus.cc:740] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e23c0ce8d89940da8eac6a6002e349c5, State: Running, Role: LEADER
I20260709 12:19:45.372730 18292 consensus_queue.cc:260] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.374928 19040 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:45.374361 18292 raft_consensus.cc:3060] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.406612 18338 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:45.409351 25958 raft_consensus.cc:2804] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:45.410032 25958 raft_consensus.cc:697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:45.411058 25958 consensus_queue.cc:237] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.418314 19039 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:45.306416 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:45.440904 18367 raft_consensus.cc:2468] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:45.442195 18292 raft_consensus.cc:2468] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:45.477247 18191 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46436 (request call id 306) took 1020 ms. Trace:
I20260709 12:19:45.480216 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with delete_type TABLET_DATA_TOMBSTONED (TS e23c0ce8d89940da8eac6a6002e349c5 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49948
I20260709 12:19:45.485865 25945 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:45.475715 18415 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:45.477718 18191 rpcz_store.cc:276] 0709 12:19:44.456529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.456685 (+   156us) service_pool.cc:224] Handling call
0709 12:19:45.477181 (+1020496us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:45.198984 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:45.199577 (+   593us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:45.199617 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:19:45.199635 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:45.200020 (+   385us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:45.200106 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:45.200144 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:45.200441 (+   297us) tablet.cc:701] Row locks acquired
0709 12:19:45.200462 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:19:45.200623 (+   161us) write_op.cc:270] Start()
0709 12:19:45.200753 (+   130us) write_op.cc:276] Timestamp: P: 1783599585200571 usec, L: 0
0709 12:19:45.200777 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:19:45.201529 (+   752us) log.cc:844] Serialized 536 byte log entry
0709 12:19:45.303425 (+101896us) op_driver.cc:464] REPLICATION: finished
0709 12:19:45.343397 (+ 39972us) write_op.cc:301] APPLY: starting
0709 12:19:45.343487 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:45.344631 (+  1144us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:45.344757 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:45.345492 (+   735us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:45.345682 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:19:45.345727 (+    45us) write_op.cc:312] APPLY: finished
0709 12:19:45.345929 (+   202us) log.cc:844] Serialized 25 byte log entry
0709 12:19:45.385705 (+ 39776us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:45.407492 (+ 21787us) write_op.cc:454] Released schema lock
0709 12:19:45.407561 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:45.407989 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:45.408020 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":63853,"reported_tablets":1,"spinlock_wait_cycles":44800,"child_traces":[["op",{"apply.queue_time_us":39686,"apply.run_cpu_time_us":3408,"apply.run_wall_time_us":64571,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8303,"num_ops":1,"prepare.queue_time_us":69215,"prepare.run_cpu_time_us":2955,"prepare.run_wall_time_us":102188,"replication_time_us":102478,"thread_start_us":108366,"threads_started":2}]]}
W20260709 12:19:45.508718 18192 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 215) took 44308 ms (client timeout 15000 ms). Trace:
I20260709 12:19:45.512666 25945 tablet_bootstrap.cc:492] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:19:45.513512 25945 ts_tablet_manager.cc:1403] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.274s	user 0.031s	sys 0.031s
W20260709 12:19:45.509243 18192 rpcz_store.cc:269] 0709 12:19:01.200868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.201496 (+   628us) service_pool.cc:224] Handling call
0709 12:19:45.508534 (+44307038us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428049,"reported_tablets":39}
W20260709 12:19:45.519898 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 48126.5us
I20260709 12:19:45.529816 25963 raft_consensus.cc:2804] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:45.546954 25945 raft_consensus.cc:359] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.547842 25945 raft_consensus.cc:740] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: FOLLOWER
I20260709 12:19:45.548779 25945 consensus_queue.cc:260] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.549821 25963 raft_consensus.cc:697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:45.560305 25992 tablet_replica.cc:333] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:45.565639 25963 consensus_queue.cc:237] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.585856 25945 ts_tablet_manager.cc:1434] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:19:45.594138 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3 received from {username='slave'} at 127.0.0.1:46956
I20260709 12:19:45.594681 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-ea246fa8cc7e4edbb136da2ee92fb1d3 on tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:45.606889 18195 catalog_manager.cc:5697] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 17 to 19, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 19 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:45.621482 18195 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:45.623221 25992 raft_consensus.cc:2243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:45.624245 25992 raft_consensus.cc:2272] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:45.627683 25992 ts_tablet_manager.cc:1916] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:19:45.570983 18196 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 216) took 17964 ms (client timeout 15000 ms). Trace:
W20260709 12:19:45.646126 18196 rpcz_store.cc:269] 0709 12:19:27.606087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.606237 (+   150us) service_pool.cc:224] Handling call
0709 12:19:27.932975 (+326738us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54003ebf54. stack: 00007f3c65d5d343 00007f3c65d5d240 00007f3c656ec1d4 00007f3c6f42a304 00007f3c6f42a299 00007f3c68de6090 00007f3c68e84f3a 00007f3c6ec3ae9c 00007f3c6ed3904e 00007f3c6d1fa2a4 00007f3c6d1fa232 00007f3c6d1fa161 00007f3c6d1fa0dc 00007f3c6d1f9392 00007f3c671b748c 00007f3c671b68c6
0709 12:19:45.570781 (+17637806us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55466,"reported_tablets":39,"spinlock_wait_cycles":634457472}
I20260709 12:19:45.689924 18192 catalog_manager.cc:5697] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 19, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:45.696664 25992 ts_tablet_manager.cc:1929] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 12:19:45.697233 25992 log.cc:1199] T ea246fa8cc7e4edbb136da2ee92fb1d3 P e23c0ce8d89940da8eac6a6002e349c5: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/wals/ea246fa8cc7e4edbb136da2ee92fb1d3
I20260709 12:19:45.706758 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:19:45.720538 18371 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:45.722366 25963 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:45.730839 18193 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 reported cstate change: term changed from 20 to 21, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65). New cstate: current_term: 21 leader_uuid: "e23c0ce8d89940da8eac6a6002e349c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:45.744637 18195 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41430 (request call id 217) took 2918 ms. Trace:
I20260709 12:19:45.753007 19039 consensus_queue.cc:237] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:45.745046 18195 rpcz_store.cc:276] 0709 12:19:42.826409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.826540 (+   131us) service_pool.cc:224] Handling call
0709 12:19:45.677561 (+2851021us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:45.677593 (+    32us) write_op.cc:454] Released schema lock
0709 12:19:45.744572 (+ 66979us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:45.667542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:45.668114 (+   572us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:45.668158 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:19:45.668181 (+    23us) tablet.cc:662] Decoding operations
0709 12:19:45.668598 (+   417us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:45.668665 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:45.668697 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:19:45.669045 (+   348us) tablet.cc:701] Row locks acquired
0709 12:19:45.669069 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:19:45.669209 (+   140us) write_op.cc:270] Start()
0709 12:19:45.669326 (+   117us) write_op.cc:276] Timestamp: P: 1783599585669161 usec, L: 0
0709 12:19:45.669350 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:19:45.670039 (+   689us) log.cc:844] Serialized 919 byte log entry
0709 12:19:45.672230 (+  2191us) op_driver.cc:464] REPLICATION: finished
0709 12:19:45.672683 (+   453us) write_op.cc:301] APPLY: starting
0709 12:19:45.672775 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:45.674209 (+  1434us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:45.674336 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:45.675728 (+  1392us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:45.675932 (+   204us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:19:45.675989 (+    57us) write_op.cc:312] APPLY: finished
0709 12:19:45.676202 (+   213us) log.cc:844] Serialized 33 byte log entry
0709 12:19:45.676375 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:45.676472 (+    97us) write_op.cc:454] Released schema lock
0709 12:19:45.676515 (+    43us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":1780,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":258,"apply.run_cpu_time_us":4124,"apply.run_wall_time_us":4124,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8352,"num_ops":2,"prepare.queue_time_us":19542,"prepare.run_cpu_time_us":2821,"prepare.run_wall_time_us":2822,"replication_time_us":2761}]]}
I20260709 12:19:45.774260 18367 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:45.776005 25616 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:45.778369 18194 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46444 (request call id 299) took 1516 ms. Trace:
I20260709 12:19:45.779129 18194 rpcz_store.cc:276] 0709 12:19:44.262113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.262272 (+   159us) service_pool.cc:224] Handling call
0709 12:19:45.778309 (+1516037us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":10}
I20260709 12:19:45.785516 18293 raft_consensus.cc:1275] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:45.787067 25616 consensus_queue.cc:1048] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:45.840451 25955 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index 8 to 11, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:45.845223 18285 tablet_service.cc:1558] Processing DeleteTablet for tablet ea246fa8cc7e4edbb136da2ee92fb1d3 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 10 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:49948
I20260709 12:19:45.846777 18195 catalog_manager.cc:5697] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 17 to 18, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } }
I20260709 12:19:45.847846 18182 catalog_manager.cc:5028] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): tablet ea246fa8cc7e4edbb136da2ee92fb1d3 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:19:45.852656 18292 raft_consensus.cc:1275] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:45.857347 25974 consensus_queue.cc:1048] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:45.859473 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8903d70080d24236be955433a92791c7 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:45.858856 18293 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 8 to 11, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:45.859952 18374 raft_consensus.cc:2955] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 8 to 11, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:45.886639 18194 catalog_manager.cc:5697] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 18 to 19, leader changed from 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 19 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:45.892349 18195 catalog_manager.cc:5697] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 8 to 11, VOTER 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67) evicted. New cstate: current_term: 17 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:46.005204 18438 tablet_service.cc:1558] Processing DeleteTablet for tablet 8903d70080d24236be955433a92791c7 with delete_type TABLET_DATA_TOMBSTONED (TS 177254018ba7448aa2b9110c9c9be9e3 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:44010
I20260709 12:19:46.022264 18300 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:46.028172 25958 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:46.037228 19039 raft_consensus.cc:1275] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:46.058234 25702 consensus_queue.cc:1048] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:19:46.065619 18371 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:46.074407 19039 raft_consensus.cc:1275] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:46.073251 26004 tablet_replica.cc:333] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:19:46.077451 25986 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:46.078934 25986 consensus_queue.cc:1048] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:46.097710 19044 raft_consensus.cc:2955] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 10 to 14, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:46.142158 26004 raft_consensus.cc:2243] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:46.143190 26004 raft_consensus.cc:2272] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:46.160467 26004 ts_tablet_manager.cc:1916] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:46.251340 26004 ts_tablet_manager.cc:1929] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.10
I20260709 12:19:46.251722 26004 log.cc:1199] T 8903d70080d24236be955433a92791c7 P 177254018ba7448aa2b9110c9c9be9e3: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/wals/8903d70080d24236be955433a92791c7
I20260709 12:19:46.253348 18183 catalog_manager.cc:5028] TS 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): tablet 8903d70080d24236be955433a92791c7 (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
I20260709 12:19:46.681212 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:46.824519 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:46.832451 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:46.833272 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:46.948975 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:47.022291 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
W20260709 12:19:47.030241 18338 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:19:47.074781 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.115414 18338 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.156342 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 44056.1us
W20260709 12:19:47.168665 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.214241 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.216687 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.217859 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.219413 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:19:47.221122 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.225628 17157 tablet_replica.cc:333] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.226702 17157 raft_consensus.cc:2243] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.227383 17157 raft_consensus.cc:2272] T 7dd86332c91f4c8d8616b22b5daaf743 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.227890 18441 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:47.230257 17157 tablet_replica.cc:333] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.231254 17157 raft_consensus.cc:2243] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.232436 17157 raft_consensus.cc:2272] T 1d0efe89255942c4bd576ff4b96cd91d P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.261291 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.277557 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.291813 17157 tablet_replica.cc:333] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.299903 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.300110 17157 raft_consensus.cc:2243] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:19:47.294162 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72e8da9e5fd9449f93381781e125fada. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.296413 18371 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:19:47.292819 18266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 45935us
I20260709 12:19:47.302079 25986 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:47.308341 17157 raft_consensus.cc:2272] T 83652a40670f4224a56a5a1347b9d1a1 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.313114 17157 tablet_replica.cc:333] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.320393 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.321601 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.322891 17157 raft_consensus.cc:2243] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.324605 17157 raft_consensus.cc:2272] T 0936d54d511a4cc3903264974a56ecb7 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.328159 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.331959 17157 tablet_replica.cc:333] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.322997 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.335448 17157 raft_consensus.cc:2243] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.339807 17157 raft_consensus.cc:2272] T 9e2ee63a449648bfb064fadee177cadb P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.342823 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.343746 17157 tablet_replica.cc:333] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.346266 17157 raft_consensus.cc:2243] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:19:47.360702 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.380282 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.380988 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.388391 17157 raft_consensus.cc:2272] T e7d0369b4aea4afb960bf859e9b7738c P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.390285 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.392861 17157 tablet_replica.cc:333] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.393724 17157 raft_consensus.cc:2243] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:19:47.396626 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.399299 17157 raft_consensus.cc:2272] T 9934c84956844d20a6258c5dca3833c6 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.399513 25986 raft_consensus.cc:2955] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:19:47.404361 17157 tablet_replica.cc:333] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.405365 17157 raft_consensus.cc:2243] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.406490 17157 raft_consensus.cc:2272] T c749b52c1c8c44b78cee98b4b4d2ef29 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.413225 18374 raft_consensus.cc:2955] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:19:47.431672 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.422981 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.442500 17157 tablet_replica.cc:333] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.448882 17157 raft_consensus.cc:2243] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.449704 17157 raft_consensus.cc:2272] T 8903d70080d24236be955433a92791c7 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.461016 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.472450 17157 tablet_replica.cc:333] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.481526 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.483259 17157 raft_consensus.cc:2243] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.484575 17157 raft_consensus.cc:2272] T 9fc8b80684174ecb8c933560e8143c49 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.492913 17157 tablet_replica.cc:333] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.494302 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.493903 18192 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 12, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:19:47.500187 17157 raft_consensus.cc:2243] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.514662 17157 raft_consensus.cc:2272] T 2e4622d18b474b659ebea19e9aad0d05 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.517570 17157 tablet_replica.cc:333] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.518724 17157 raft_consensus.cc:2243] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.522529 17157 raft_consensus.cc:2272] T 5d4d1763576e4c49849611360f37665a P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:19:47.536782 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.537568 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.546255 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.559698 17157 tablet_replica.cc:333] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
W20260709 12:19:47.559885 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.561318 17157 raft_consensus.cc:2243] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.563797 18374 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:47.565275 17157 raft_consensus.cc:2272] T 977c0af9c8e94119b3d18a0ee7401458 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.571092 17157 tablet_replica.cc:333] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.576922 17157 raft_consensus.cc:2243] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.577641 17157 raft_consensus.cc:2272] T 67a1783f49244bd5ba98721205d1ff6e P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.580716 17157 tablet_replica.cc:333] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.581570 17157 raft_consensus.cc:2243] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.582967 17157 raft_consensus.cc:2272] T 01de3fbc59f243ba8a35edaa5fcfe86f P e23c0ce8d89940da8eac6a6002e349c5 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.586790 17157 tablet_replica.cc:333] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.593840 18441 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:19:47.594933 18338 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821): Couldn't send request to peer 991dcd2a70c74fdb89ca9494cc09fb7b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d0efe89255942c4bd576ff4b96cd91d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.595536 26022 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:19:47.600528 18338 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.601751 17157 raft_consensus.cc:2243] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:19:47.633992 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.637964 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.639081 17157 raft_consensus.cc:2272] T a971a418238d45d288b49960c0abeea0 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.645860 17157 tablet_replica.cc:333] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.647418 17157 raft_consensus.cc:2243] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.649263 17157 raft_consensus.cc:2272] T b4688e522b8b48f7bb98e0832680f9e2 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.652114 17157 tablet_replica.cc:333] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.653721 17157 raft_consensus.cc:2243] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.654390 17157 raft_consensus.cc:2272] T e98a95c8e3c14bcfa51b054f4829293b P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.656805 25702 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
I20260709 12:19:47.659004 18445 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } }
W20260709 12:19:47.679946 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet c1f155085cb647ff957b468384227872: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:19:47.683388 18246 auto_rebalancer.cc:629] Failed to schedule move for tablet 1a92f90eb0424e3d870764a2a328e508: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:47.701763 17157 tablet_replica.cc:333] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.698835 18192 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index -1 to 13, NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) added. New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:47.702813 17157 raft_consensus.cc:2243] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:47.708241 17157 raft_consensus.cc:2272] T fafcfc4614304a81b0eadc8f445bfdb8 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.750963 17157 tablet_replica.cc:333] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.770612 17157 raft_consensus.cc:2243] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.797024 17157 raft_consensus.cc:2272] T d8bf6d66a261490caa69a80b24ea015f P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.848304 26032 ts_tablet_manager.cc:933] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:47.854317 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 58472.4us
I20260709 12:19:47.865226 17157 tablet_replica.cc:333] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.866516 17157 raft_consensus.cc:2243] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.915925 17157 raft_consensus.cc:2272] T 1a92f90eb0424e3d870764a2a328e508 P e23c0ce8d89940da8eac6a6002e349c5 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.920640 17157 tablet_replica.cc:333] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.924125 17157 raft_consensus.cc:2243] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:19:47.925850 26019 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:47.949705 26019 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:47.952853 26019 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:47.953343 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:47.960219 26032 tablet_copy_client.cc:323] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.67:40227
I20260709 12:19:47.955555 17157 raft_consensus.cc:2272] T c1f155085cb647ff957b468384227872 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:47.982214 17157 tablet_replica.cc:333] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:47.983202 17157 raft_consensus.cc:2243] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:19:47.997467 18263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 44565.2us
I20260709 12:19:47.999755 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57dbd4858e8f41a2af67d6de6eae9928" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:19:48.001610 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 146122us
I20260709 12:19:48.005138 17157 raft_consensus.cc:2272] T 53c08deb8e86480a878b17c55d6bc089 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.003782 18263 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:48.013882 26039 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:48.014046 18459 tablet_copy_service.cc:140] P 177254018ba7448aa2b9110c9c9be9e3: Received BeginTabletCopySession request for tablet 72e8da9e5fd9449f93381781e125fada from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:46956)
I20260709 12:19:48.014703 18459 tablet_copy_service.cc:161] P 177254018ba7448aa2b9110c9c9be9e3: Beginning new tablet copy session on tablet 72e8da9e5fd9449f93381781e125fada from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:46956: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-72e8da9e5fd9449f93381781e125fada
I20260709 12:19:48.024719 18459 tablet_copy_source_session.cc:215] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:48.029721 17157 tablet_replica.cc:333] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.052145 17157 raft_consensus.cc:2243] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.052865 17157 raft_consensus.cc:2272] T 85f32970b7374dcf9b9bac47791c71e3 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.055716 17157 tablet_replica.cc:333] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.062729 26032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e8da9e5fd9449f93381781e125fada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:48.080725 17157 raft_consensus.cc:2243] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.082001 26039 raft_consensus.cc:493] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:48.087720 17157 raft_consensus.cc:2272] T 72e8da9e5fd9449f93381781e125fada P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.087615 26039 raft_consensus.cc:515] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.089978 26039 leader_election.cc:290] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.094173 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.100044 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1686d6e6b8ae45aab8c5d9d916f8485d" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.101631 18266 leader_election.cc:304] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:48.102998 26039 raft_consensus.cc:2749] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.123538 17157 tablet_replica.cc:333] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.124405 17157 raft_consensus.cc:2243] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.125152 17157 raft_consensus.cc:2272] T 57dbd4858e8f41a2af67d6de6eae9928 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.146713 17157 tablet_replica.cc:333] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.147612 17157 raft_consensus.cc:2243] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.148231 17157 raft_consensus.cc:2272] T b81b910b60b0422f9db4c0164608fcbb P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.155239 26032 tablet_copy_client.cc:806] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:48.155916 26032 tablet_copy_client.cc:670] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:48.186602 26032 tablet_copy_client.cc:538] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:48.188277 26041 ts_tablet_manager.cc:933] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Initiating tablet copy from peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.151147 17157 tablet_replica.cc:333] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.214082 26041 tablet_copy_client.cc:323] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Beginning tablet copy session from remote peer at address 127.16.193.66:44577
I20260709 12:19:48.216122 17157 raft_consensus.cc:2243] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.217193 18385 tablet_copy_service.cc:140] P 3da7ad8fa20a4591963b660e4c848e7f: Received BeginTabletCopySession request for tablet 1d0efe89255942c4bd576ff4b96cd91d from peer 991dcd2a70c74fdb89ca9494cc09fb7b ({username='slave'} at 127.0.0.1:56140)
I20260709 12:19:48.223642 18385 tablet_copy_service.cc:161] P 3da7ad8fa20a4591963b660e4c848e7f: Beginning new tablet copy session on tablet 1d0efe89255942c4bd576ff4b96cd91d from peer 991dcd2a70c74fdb89ca9494cc09fb7b at {username='slave'} at 127.0.0.1:56140: session id = 991dcd2a70c74fdb89ca9494cc09fb7b-1d0efe89255942c4bd576ff4b96cd91d
I20260709 12:19:48.223706 17157 raft_consensus.cc:2272] T 80bc05a330c64471aaa556af75bea031 P e23c0ce8d89940da8eac6a6002e349c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.231910 18385 tablet_copy_source_session.cc:215] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:48.235357 26041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0efe89255942c4bd576ff4b96cd91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:48.236744 17157 tablet_replica.cc:333] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.237663 17157 raft_consensus.cc:2243] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.238425 17157 raft_consensus.cc:2272] T 8e62e9fad92b449d9315da4aa77082b4 P e23c0ce8d89940da8eac6a6002e349c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.251690 25950 raft_consensus.cc:493] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:48.252285 25950 raft_consensus.cc:515] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.255836 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.257930 25950 leader_election.cc:290] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.259909 26032 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:19:48.264140 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9f2e3c205c14e43b532c52bc7fa4c90" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:48.266464 18263 leader_election.cc:304] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:48.270524 26039 raft_consensus.cc:2749] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:48.277595 17157 tablet_replica.cc:333] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.278486 17157 raft_consensus.cc:2243] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.279663 17157 raft_consensus.cc:2272] T 389aa517d744435bb043ff8c52d86c2d P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.296970 26039 raft_consensus.cc:493] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:48.297546 26039 raft_consensus.cc:515] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.299746 26039 leader_election.cc:290] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.301612 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.303360 26040 raft_consensus.cc:493] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:48.303918 26040 raft_consensus.cc:515] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.305498 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc5f8db079f74a52b906175ad2c4cb56" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:48.306041 26040 leader_election.cc:290] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.308640 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.313084 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.314251 17157 tablet_replica.cc:333] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.316115 18263 leader_election.cc:304] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:48.318222 26040 raft_consensus.cc:2749] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:48.319541 17157 raft_consensus.cc:2243] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.320626 17157 raft_consensus.cc:2272] T 3153d259c2254b2d9bdc71d8322cafc0 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.323802 26033 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.324456 26033 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.326857 26033 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.330897 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:48.301651 26041 tablet_copy_client.cc:806] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:48.339792 25950 raft_consensus.cc:493] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:48.340409 25950 raft_consensus.cc:515] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.342195 17157 tablet_replica.cc:333] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.342661 25950 leader_election.cc:290] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.343019 17157 raft_consensus.cc:2243] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.343760 17157 raft_consensus.cc:2272] T 7d788f8182ca4b13bddd2acdf511873f P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.346110 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.349027 18264 leader_election.cc:304] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:48.349807 25950 raft_consensus.cc:2749] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:48.355706 17157 tablet_replica.cc:333] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.356568 17157 raft_consensus.cc:2243] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.383559 17157 raft_consensus.cc:2272] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.384164 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5d2fad2696483cae18eab3ba337ec9" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.392197 18264 leader_election.cc:304] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:19:48.391322 18338 leader_election.cc:336] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.393234 18338 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.393517 26040 raft_consensus.cc:2749] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.395265 26033 raft_consensus.cc:2749] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:48.396366 26040 raft_consensus.cc:493] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 991dcd2a70c74fdb89ca9494cc09fb7b)
I20260709 12:19:48.396916 26040 raft_consensus.cc:515] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.400559 26039 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:19:48.401085 26039 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.400967 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:48.403219 26039 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.404963 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.378003 26041 tablet_copy_client.cc:670] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Starting download of 1 WAL segments...
W20260709 12:19:48.409940 25950 raft_consensus.cc:670] T adea31c273544d04b7f16f9b9a3ae610 P e23c0ce8d89940da8eac6a6002e349c5: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 12:19:48.411212 26039 raft_consensus.cc:493] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:19:48.410315 26040 leader_election.cc:290] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.417198 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "548b719daf794fb881c5606ebec2abb3" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.418990 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbd14f2572940d797f0297b94365af4" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.420722 26041 tablet_copy_client.cc:538] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:48.413655 26039 raft_consensus.cc:515] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.431619 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.439057 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53e315b86a94adb89240687e1a622ed" candidate_uuid: "e23c0ce8d89940da8eac6a6002e349c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.444042 26039 leader_election.cc:290] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.446620 18266 leader_election.cc:304] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:48.446141 17157 tablet_replica.cc:333] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.447940 17157 raft_consensus.cc:2243] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.448891 18266 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:48.449636 26040 raft_consensus.cc:2749] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:48.450304 26040 raft_consensus.cc:2706] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Received pre-election callback for term 24 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:19:48.451162 17157 raft_consensus.cc:2272] T 2cbd14f2572940d797f0297b94365af4 P e23c0ce8d89940da8eac6a6002e349c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.453823 17157 tablet_replica.cc:333] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.454521 17157 raft_consensus.cc:2243] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.455591 18266 leader_election.cc:304] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e23c0ce8d89940da8eac6a6002e349c5; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:19:48.457970 26039 raft_consensus.cc:2749] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:19:48.461410 26041 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap starting.
I20260709 12:19:48.461755 17157 raft_consensus.cc:2272] T 6c5d2fad2696483cae18eab3ba337ec9 P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.487002 17157 tablet_replica.cc:333] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.491185 17157 raft_consensus.cc:2243] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.499213 17157 raft_consensus.cc:2272] T 548b719daf794fb881c5606ebec2abb3 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.511246 17157 tablet_replica.cc:333] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.534226 26032 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:48.535219 26032 tablet_bootstrap.cc:492] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:19:48.535970 26032 ts_tablet_manager.cc:1403] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.276s	user 0.045s	sys 0.004s
I20260709 12:19:48.512087 17157 raft_consensus.cc:2243] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.543332 17157 raft_consensus.cc:2272] T 80f93a4f5d0e45dd8b049fe76d3a0fda P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.565201 17157 tablet_replica.cc:333] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.568967 17157 raft_consensus.cc:2243] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.569702 17157 raft_consensus.cc:2272] T b9f2e3c205c14e43b532c52bc7fa4c90 P e23c0ce8d89940da8eac6a6002e349c5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.572798 17157 tablet_replica.cc:333] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.574075 17157 raft_consensus.cc:2243] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.574738 17157 raft_consensus.cc:2272] T 1686d6e6b8ae45aab8c5d9d916f8485d P e23c0ce8d89940da8eac6a6002e349c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.577775 17157 tablet_replica.cc:333] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.578781 17157 raft_consensus.cc:2243] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.579486 17157 raft_consensus.cc:2272] T f53e315b86a94adb89240687e1a622ed P e23c0ce8d89940da8eac6a6002e349c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.582037 17157 tablet_replica.cc:333] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.582933 17157 raft_consensus.cc:2243] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.584263 17157 raft_consensus.cc:2272] T ad852e5220a14f4898d2f069a5fb4a63 P e23c0ce8d89940da8eac6a6002e349c5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.571110 26032 raft_consensus.cc:359] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:48.586149 26032 raft_consensus.cc:740] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:19:48.589413 17157 tablet_replica.cc:333] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.590256 17157 raft_consensus.cc:2243] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.592811 17157 raft_consensus.cc:2272] T 6ab0a3f9b34149918a57aec5d20b4884 P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.586844 26032 consensus_queue.cc:260] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:48.623261 26032 ts_tablet_manager.cc:1434] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.087s	user 0.005s	sys 0.002s
I20260709 12:19:48.630106 18459 tablet_copy_service.cc:342] P 177254018ba7448aa2b9110c9c9be9e3: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-72e8da9e5fd9449f93381781e125fada received from {username='slave'} at 127.0.0.1:46956
I20260709 12:19:48.630573 18459 tablet_copy_service.cc:434] P 177254018ba7448aa2b9110c9c9be9e3: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-72e8da9e5fd9449f93381781e125fada on tablet 72e8da9e5fd9449f93381781e125fada with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:48.630687 17157 tablet_replica.cc:333] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.631502 17157 raft_consensus.cc:2243] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.632572 17157 raft_consensus.cc:2272] T e9decd1c7d7f475b8890a8519847dca5 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.635260 17157 tablet_replica.cc:333] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.635987 17157 raft_consensus.cc:2243] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.636629 17157 raft_consensus.cc:2272] T ca383fa1243547c3b32dfbd6857a3fba P e23c0ce8d89940da8eac6a6002e349c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.665624 19040 raft_consensus.cc:1217] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 12:19:48.680264 17157 tablet_replica.cc:333] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.681146 17157 raft_consensus.cc:2243] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:19:48.700363 17157 raft_consensus.cc:2272] T 0071c33745ae42e48eca14d86aa5fac6 P e23c0ce8d89940da8eac6a6002e349c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.707774 26035 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.728370 17157 tablet_replica.cc:333] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5: stopping tablet replica
I20260709 12:19:48.729166 17157 raft_consensus.cc:2243] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:48.729744 17157 raft_consensus.cc:2272] T cc5f8db079f74a52b906175ad2c4cb56 P e23c0ce8d89940da8eac6a6002e349c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:48.734892 26035 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.737845 26035 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:48.740810 19006 leader_election.cc:336] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.760682 26049 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.761348 26049 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:48.770375 18338 leader_election.cc:336] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.774410 26047 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.774974 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.776070 26049 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.788230 26047 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.789489 18339 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.790620 26049 raft_consensus.cc:2749] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.791872 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.792146 26047 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:48.796181 18412 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.796773 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.798481 26047 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:48.802625 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 111665us
I20260709 12:19:48.804860 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.805471 18367 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:19:48.811733 26012 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.812319 26012 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.814555 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.815269 26012 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:48.815474 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:48.816452 26038 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:48.816871 26038 raft_consensus.cc:493] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.817250 26038 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:48.818603 19006 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.819095 19006 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.825448 26035 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:48.826236 26035 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.826644 26035 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.847249 26047 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:48.848112 19006 leader_election.cc:336] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.847959 26047 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.849560 26046 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.850281 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.849254 26035 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.850893 18374 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:48.852310 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.850080 26046 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:48.853394 18412 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.855003 26046 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.856786 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:48.858673 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.859117 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.859937 26035 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:48.860427 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:48.860409 26035 raft_consensus.cc:493] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.861228 26035 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:48.861248 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:48.864151 26020 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:48.866263 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.868692 26048 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:48.869078 26048 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.869465 26048 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:48.871855 26038 raft_consensus.cc:515] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.875093 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c749b52c1c8c44b78cee98b4b4d2ef29" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:48.875646 18374 raft_consensus.cc:3060] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:48.878877 19006 leader_election.cc:336] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.881084 26047 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.882215 18374 raft_consensus.cc:2468] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
I20260709 12:19:48.883391 19009 leader_election.cc:304] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.884281 26053 raft_consensus.cc:2804] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:48.884774 26053 raft_consensus.cc:697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:48.885609 26053 consensus_queue.cc:237] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.887871 26047 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.888891 26047 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:48.895699 18412 leader_election.cc:336] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.901489 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:48.903540 18374 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
I20260709 12:19:48.903788 26035 raft_consensus.cc:515] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.890722 26048 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.904631 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.905465 26046 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:48.905834 26046 raft_consensus.cc:493] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.906286 26046 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.906672 26048 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:48.908636 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0936d54d511a4cc3903264974a56ecb7" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:48.897691 26038 leader_election.cc:290] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.909314 18367 raft_consensus.cc:3060] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:48.910524 18412 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.911861 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d0369b4aea4afb960bf859e9b7738c" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:48.912452 19040 raft_consensus.cc:3060] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:48.925302 26041 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:19:48.926192 19006 leader_election.cc:336] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.928435 26047 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.934410 19040 raft_consensus.cc:2468] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 23.
I20260709 12:19:48.935864 18412 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.937594 26048 raft_consensus.cc:2804] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:48.905987 26035 leader_election.cc:290] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:48.938293 26048 raft_consensus.cc:697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:48.926383 26041 tablet_bootstrap.cc:492] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Bootstrap complete.
I20260709 12:19:48.940181 26048 consensus_queue.cc:237] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.947793 18367 raft_consensus.cc:2468] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:48.960261 19009 leader_election.cc:304] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:48.961184 26053 raft_consensus.cc:2804] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:48.940042 26046 raft_consensus.cc:515] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:48.961715 26053 raft_consensus.cc:697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:48.967669 26046 leader_election.cc:290] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:19:48.968016 18412 leader_election.cc:336] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:48.968894 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01de3fbc59f243ba8a35edaa5fcfe86f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:48.969475 18367 raft_consensus.cc:3060] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.962689 26053 consensus_queue.cc:237] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:48.986653 18193 catalog_manager.cc:5697] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 22 to 23, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 23 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.004222 26033 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:48.940986 18194 catalog_manager.cc:5697] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 19 to 20, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 20 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:48.941659 26041 ts_tablet_manager.cc:1403] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent bootstrapping tablet: real 0.480s	user 0.053s	sys 0.002s
I20260709 12:19:49.004853 26033 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.008234 18367 raft_consensus.cc:2468] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 17.
I20260709 12:19:49.009491 18415 leader_election.cc:304] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.009686 26033 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:49.012050 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:49.012708 18442 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 18.
I20260709 12:19:49.011144 26046 raft_consensus.cc:2804] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:49.014254 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:49.013432 26046 raft_consensus.cc:697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:49.018224 26050 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.018676 26050 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.019105 26050 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:49.022702 18338 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.019071 26046 consensus_queue.cc:237] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.027345 26050 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.032043 26041 raft_consensus.cc:359] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:49.033034 26041 raft_consensus.cc:740] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Initialized, Role: LEARNER
I20260709 12:19:49.033803 26041 consensus_queue.cc:260] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } }
I20260709 12:19:49.044541 26050 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:49.055572 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc8b80684174ecb8c933560e8143c49" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:49.056173 18442 raft_consensus.cc:3060] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.076889 26052 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.077783 26052 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.083393 26052 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:49.085031 18442 raft_consensus.cc:2468] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 19.
I20260709 12:19:49.094234 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:49.088551 26041 ts_tablet_manager.cc:1434] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
W20260709 12:19:49.104193 18338 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.104351 18385 tablet_copy_service.cc:342] P 3da7ad8fa20a4591963b660e4c848e7f: Request end of tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-1d0efe89255942c4bd576ff4b96cd91d received from {username='slave'} at 127.0.0.1:56140
I20260709 12:19:49.105083 18385 tablet_copy_service.cc:434] P 3da7ad8fa20a4591963b660e4c848e7f: ending tablet copy session 991dcd2a70c74fdb89ca9494cc09fb7b-1d0efe89255942c4bd576ff4b96cd91d on tablet 1d0efe89255942c4bd576ff4b96cd91d with peer 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:19:49.112150 18339 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.114391 26033 raft_consensus.cc:2804] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:49.117439 26033 raft_consensus.cc:697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:49.114388 19006 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.118769 19006 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.119076 26033 consensus_queue.cc:237] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.120537 26038 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.127993 26034 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.128597 26034 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.131014 26034 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:49.140715 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:19:49.141592 18338 leader_election.cc:336] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.150938 18339 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.151854 26049 raft_consensus.cc:2749] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.167343 26046 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.176398 26046 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.181530 26046 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:49.191766 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:49.192433 19040 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:19:49.198619 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:49.200105 26048 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:49.200502 26048 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.200853 26048 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:49.209523 18412 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:19:49.220016 26061 log.cc:927] Time spent T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:49.229455 19040 raft_consensus.cc:1217] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEARNER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
I20260709 12:19:49.212647 26048 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.237697 18192 catalog_manager.cc:5697] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 18 to 19, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.242291 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bf6d66a261490caa69a80b24ea015f" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:49.242843 19040 raft_consensus.cc:3060] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:49.236500 26048 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:49.269549 19040 raft_consensus.cc:2468] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:19:49.282094 26046 raft_consensus.cc:1064] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:19:49.282166 18412 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:49.283327 26048 raft_consensus.cc:2804] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:49.283798 26048 raft_consensus.cc:697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:49.284909 26048 consensus_queue.cc:237] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.285061 26046 consensus_queue.cc:237] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:49.292474 18412 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.303524 18193 catalog_manager.cc:5697] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 16 to 17, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 17 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:49.311978 18195 catalog_manager.cc:5697] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 21 to 22, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:49.316797 19040 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEARNER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:49.319149 18367 raft_consensus.cc:1275] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:49.319216 26046 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:49.324031 26050 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.324051 26046 consensus_queue.cc:1048] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:49.324759 26050 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:49.327522 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:49.329149 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:49.329733 18442 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 13.
I20260709 12:19:49.342451 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:49.344527 26034 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:49.344996 26034 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.345393 26034 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:49.369035 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:49.327013 26050 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:49.380573 18338 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.388015 26046 raft_consensus.cc:2955] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:49.401593 18367 raft_consensus.cc:2955] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:49.400564 18193 catalog_manager.cc:5697] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 19 to 20, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 20 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.422290 26052 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.422907 26052 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.430678 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:49.431725 26052 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:49.438730 19006 leader_election.cc:336] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.439201 19006 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.440909 26053 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.415705 26034 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.460883 26034 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:49.461829 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a92f90eb0424e3d870764a2a328e508" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:49.462447 18442 raft_consensus.cc:3060] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:49.462520 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:49.471664 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:49.474967 18193 catalog_manager.cc:5697] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 12 to 13, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:49.484187 18367 raft_consensus.cc:1275] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:19:49.484469 25974 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.485057 25974 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.490257 26038 consensus_queue.cc:1048] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:19:49.495462 25974 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:49.496191 18374 raft_consensus.cc:1275] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:19:49.498816 26038 consensus_queue.cc:1048] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:49.506970 19044 raft_consensus.cc:1275] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:19:49.508960 26048 consensus_queue.cc:1048] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:19:49.518163 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:49.519032 18412 leader_election.cc:336] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.533480 18442 raft_consensus.cc:2468] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 14.
I20260709 12:19:49.536458 18339 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:49.538141 26049 raft_consensus.cc:2804] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:49.546703 18442 raft_consensus.cc:1275] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:49.538645 26049 raft_consensus.cc:697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
I20260709 12:19:49.578969 18367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:49.579636 18367 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 15.
I20260709 12:19:49.584791 26049 consensus_queue.cc:237] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:49.586236 18374 raft_consensus.cc:1275] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:49.575091 26020 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.572419 26034 consensus_queue.cc:1048] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 12:19:49.635434 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55197.3us
I20260709 12:19:49.587383 26020 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.649010 19044 raft_consensus.cc:2955] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:49.668416 26069 log.cc:927] Time spent T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:19:49.668960 26054 log.cc:927] Time spent T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:49.703860 26078 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.705619 26078 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:49.706525 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:49.708725 26078 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:49.712275 26020 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:49.715117 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:19:49.716475 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:49.742522 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:49.770650 18374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
W20260709 12:19:49.772858 18412 leader_election.cc:336] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.773326 18412 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:49.775458 18338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 139165us
W20260709 12:19:49.776142 18338 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:49.781476 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:49.787086 26053 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.794617 26083 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.795550 26046 raft_consensus.cc:2749] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.799337 26083 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.797843 26053 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.780562 26047 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.819001 18195 catalog_manager.cc:5697] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 13 to 14, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 14 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.821893 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:49.845893 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 73048us
W20260709 12:19:49.846911 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:49.810096 26078 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:49.817140 26083 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:49.821498 26047 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.872406 25974 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:49.889822 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 88462us
I20260709 12:19:49.876986 26092 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.883626 26083 consensus_queue.cc:1048] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:19:49.892131 25974 raft_consensus.cc:493] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:49.915073 18338 connection.cc:769] client connection to 127.16.193.65:37771 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:19:49.893366 26053 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:49.920836 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:49.901213 26048 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:49.911222 25974 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:49.947132 26047 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:49.899097 26092 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:49.958914 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:49.937664 26048 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.011534 26048 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:50.013788 26071 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.014370 26071 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:50.015372 26089 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
W20260709 12:19:50.016064 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 98401.7us
I20260709 12:19:50.021484 26089 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:50.041580 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.042462 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:50.048880 26071 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:50.055662 18412 leader_election.cc:336] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:50.058254 25974 raft_consensus.cc:515] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:19:50.061764 18412 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:50.064213 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 144800us
W20260709 12:19:50.065021 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.066637 26089 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:50.065804 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.067562 18412 leader_election.cc:336] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:19:50.068192 18412 leader_election.cc:336] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:19:50.071182 26101 raft_consensus.cc:2749] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:50.092054 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:50.092921 19039 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 18.
I20260709 12:19:50.111467 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:50.113399 26098 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.114010 26098 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.116267 26098 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:50.112329 19039 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:50.133358 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:50.134209 19040 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 20.
W20260709 12:19:50.137084 18412 connection.cc:769] client connection to 127.16.193.65:37771 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 12:19:50.145917 19006 connection.cc:769] client connection to 127.16.193.65:37771 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:19:50.149767 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:50.150435 18445 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:50.132616 26038 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.162645 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:50.163372 18446 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 15.
I20260709 12:19:50.166733 26071 raft_consensus.cc:1064] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: attempting to promote NON_VOTER 991dcd2a70c74fdb89ca9494cc09fb7b to VOTER
I20260709 12:19:50.169238 26071 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.185735 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.185369 26075 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.188097 26097 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:50.183480 26038 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.188539 26097 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.188931 26097 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.190167 26038 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:50.194496 25974 leader_election.cc:290] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:50.200304 26092 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
W20260709 12:19:50.205960 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.206419 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 59316.1us
I20260709 12:19:50.190888 26075 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:50.210811 26075 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:50.220577 18442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:50.221259 18442 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 21.
I20260709 12:19:50.221429 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.225031 26099 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:50.230306 26099 raft_consensus.cc:493] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.230669 26099 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.245119 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:50.249249 25974 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:50.249713 25974 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.250125 25974 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.257413 26097 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.259727 26097 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:50.247273 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.270800 26088 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:50.271262 26088 raft_consensus.cc:493] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.271633 26088 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.271726 25974 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.274494 25974 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:50.277165 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.277922 26079 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:50.278303 26079 raft_consensus.cc:493] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.278690 26079 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:50.284577 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.285131 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.285416 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.285748 18338 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:50.287379 26099 raft_consensus.cc:515] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.289690 26099 leader_election.cc:290] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:50.294178 26079 raft_consensus.cc:515] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.297621 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389aa517d744435bb043ff8c52d86c2d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:50.298230 18444 raft_consensus.cc:3060] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.299281 18445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:50.299947 18445 raft_consensus.cc:3060] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:50.299373 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 52128.1us
I20260709 12:19:50.301066 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:50.305160 26084 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:50.305542 26084 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.296203 26079 leader_election.cc:290] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:19:50.305977 26084 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.306563 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f" is_pre_election: true
I20260709 12:19:50.307154 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 21.
I20260709 12:19:50.308017 18369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f155085cb647ff957b468384227872" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:50.308619 18369 raft_consensus.cc:3060] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.316174 18445 raft_consensus.cc:2468] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 22.
I20260709 12:19:50.317396 19007 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.318238 18444 raft_consensus.cc:2468] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:50.318356 26098 raft_consensus.cc:2804] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:50.296132 26088 raft_consensus.cc:515] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.319455 26098 raft_consensus.cc:697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:50.320782 19007 leader_election.cc:304] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.321523 26038 raft_consensus.cc:2804] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:50.323325 26038 raft_consensus.cc:697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:50.324247 26038 consensus_queue.cc:237] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:50.325764 19006 leader_election.cc:336] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.327026 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.327534 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.327950 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.328615 26098 consensus_queue.cc:237] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.329679 18444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad852e5220a14f4898d2f069a5fb4a63" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
I20260709 12:19:50.330278 18444 raft_consensus.cc:3060] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:50.320837 26088 leader_election.cc:290] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:19:50.332737 19006 leader_election.cc:336] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.333457 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.333889 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.334419 19006 leader_election.cc:336] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.335189 19006 leader_election.cc:336] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.337949 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:19:50.338546 18446 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
I20260709 12:19:50.342587 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ab0a3f9b34149918a57aec5d20b4884" candidate_uuid: "3da7ad8fa20a4591963b660e4c848e7f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:50.343134 19040 raft_consensus.cc:3060] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.344493 26084 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.349566 19040 raft_consensus.cc:2468] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da7ad8fa20a4591963b660e4c848e7f in term 21.
W20260709 12:19:50.350242 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.350878 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.349330 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:50.352766 18445 raft_consensus.cc:1275] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:50.352918 25974 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:50.353447 25974 raft_consensus.cc:493] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e23c0ce8d89940da8eac6a6002e349c5)
I20260709 12:19:50.353820 25974 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:50.357802 18338 leader_election.cc:336] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.358589 18338 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.358708 18369 raft_consensus.cc:2468] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 16.
W20260709 12:19:50.360169 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.360637 19006 leader_election.cc:336] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.363065 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.363587 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.364303 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.366797 18444 raft_consensus.cc:2468] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 16.
I20260709 12:19:50.371692 18415 leader_election.cc:304] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:50.374738 19007 leader_election.cc:304] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.372541 18338 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.379997 19039 raft_consensus.cc:1275] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:50.381649 26083 raft_consensus.cc:2804] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:50.382165 26083 raft_consensus.cc:697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:50.385069 18412 leader_election.cc:336] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.385635 18412 leader_election.cc:336] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.385164 18195 catalog_manager.cc:5697] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 21 to 22, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 22 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:50.391551 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0071c33745ae42e48eca14d86aa5fac6" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:19:50.391938 25974 raft_consensus.cc:515] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:50.403220 25974 leader_election.cc:290] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:19:50.426316 19040 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEARNER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:19:50.426800 19006 leader_election.cc:336] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.428910 19039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3153d259c2254b2d9bdc71d8322cafc0" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:50.429445 18375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9decd1c7d7f475b8890a8519847dca5" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3da7ad8fa20a4591963b660e4c848e7f"
I20260709 12:19:50.430143 18375 raft_consensus.cc:3060] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.399792 26079 raft_consensus.cc:2804] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:50.401777 26097 raft_consensus.cc:2804] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:50.429524 19039 raft_consensus.cc:3060] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.466980 18339 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:19:50.478927 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.479686 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:50.480193 18338 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.452867 26079 raft_consensus.cc:697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:19:50.485678 18412 leader_election.cc:304] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 991dcd2a70c74fdb89ca9494cc09fb7b, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:19:50.484771 25974 consensus_queue.cc:1048] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:50.494530 18375 raft_consensus.cc:2468] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:50.463542 26050 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:19:50.501260 26084 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Requested vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:19:50.502558 26050 raft_consensus.cc:2764] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:19:50.439306 26083 consensus_queue.cc:237] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:19:50.504058 26112 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:50.513233 26079 consensus_queue.cc:237] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:50.532330 19039 raft_consensus.cc:2468] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
W20260709 12:19:50.536198 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49351.5us
W20260709 12:19:50.537330 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 51720us
I20260709 12:19:50.539906 26046 raft_consensus.cc:2749] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:50.595000 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 55615.8us
I20260709 12:19:50.622243 18415 leader_election.cc:304] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f; no voters: 
I20260709 12:19:50.630610 25974 raft_consensus.cc:2804] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:50.631246 25974 raft_consensus.cc:697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:50.638823 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 171948us
I20260709 12:19:50.641832 18412 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:19:50.643733 26089 consensus_queue.cc:1048] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:50.632323 25974 consensus_queue.cc:237] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } }
I20260709 12:19:50.612658 26097 raft_consensus.cc:697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Becoming Leader. State: Replica: 3da7ad8fa20a4591963b660e4c848e7f, State: Running, Role: LEADER
W20260709 12:19:50.689239 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:43821, user_credentials={real_user=slave}} blocked reactor thread for 47480us
I20260709 12:19:50.689422 26075 raft_consensus.cc:2804] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:50.690044 26075 raft_consensus.cc:697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
W20260709 12:19:50.691574 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.691171 26075 consensus_queue.cc:237] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:50.692073 18412 leader_election.cc:336] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.696264 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80f93a4f5d0e45dd8b049fe76d3a0fda" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:19:50.696924 19044 raft_consensus.cc:3060] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.719833 19044 raft_consensus.cc:2468] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 22.
I20260709 12:19:50.720490 18442 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:50.729058 18412 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
W20260709 12:19:50.740305 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 65268.1us
I20260709 12:19:50.736629 26050 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:19:50.744093 26123 raft_consensus.cc:2804] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:50.754154 26123 raft_consensus.cc:697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:19:50.755192 26123 consensus_queue.cc:237] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:50.782637 18338 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:50.782658 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.788887 18412 leader_election.cc:336] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:50.831578 18442 raft_consensus.cc:1275] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:50.834177 18446 raft_consensus.cc:1275] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:19:50.840548 18193 catalog_manager.cc:5697] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 15 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 16 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:50.847525 26079 consensus_queue.cc:1048] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:19:50.849619 26104 consensus_queue.cc:1048] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:50.788054 26097 consensus_queue.cc:237] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:50.858703 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 69174.7us
W20260709 12:19:50.865480 18412 leader_election.cc:336] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.866386 18412 leader_election.cc:336] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:50.868395 26131 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:19:50.868647 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.869310 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:50.871455 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:50.877400 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:50.890563 18195 catalog_manager.cc:5697] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 15 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:50.932471 18195 catalog_manager.cc:5697] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 15 to 16, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 16 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:50.984309 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:51.027400 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:51.029614 18195 catalog_manager.cc:5697] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: term changed from 20 to 21, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66). New cstate: current_term: 21 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.042578 26071 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:51.056891 18193 catalog_manager.cc:5697] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 18 to 19, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 19 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.058888 18193 catalog_manager.cc:5697] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 21 to 22, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 22 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.062247 18440 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:51.065595 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:51.085001 19039 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:51.111063 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:51.117779 18194 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: config changed from index 13 to 14, 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } attrs { replace: true } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:51.123168 18374 raft_consensus.cc:1275] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:51.060479 18193 catalog_manager.cc:5697] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 21 to 22, leader changed from e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 22 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.132589 26101 consensus_queue.cc:1048] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 12:19:51.171569 17157 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18321)
I20260709 12:19:51.185850 18440 raft_consensus.cc:1275] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:51.190917 26125 consensus_queue.cc:1048] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:19:51.212654 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:51.216698 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:51.231922 19039 raft_consensus.cc:1275] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:51.236315 18375 raft_consensus.cc:1275] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:19:51.235324 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:51.239928 26100 consensus_queue.cc:1048] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:51.240919 26101 consensus_queue.cc:1048] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:51.346720 19040 raft_consensus.cc:1275] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:51.348455 26100 consensus_queue.cc:1048] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:51.357375 18369 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:19:51.363124 18442 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:51.366160 26118 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:51.372713 19044 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:51.375037 26097 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:51.386905 18442 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:51.388583 26118 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:51.403990 19039 raft_consensus.cc:2955] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:51.428172 18185 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d0efe89255942c4bd576ff4b96cd91d with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:19:51.451530 26153 log.cc:927] Time spent T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.095s	user 0.005s	sys 0.000s
I20260709 12:19:51.462826 18193 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f reported cstate change: config changed from index 14 to 15, VOTER e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65) evicted. New cstate: current_term: 19 leader_uuid: "3da7ad8fa20a4591963b660e4c848e7f" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.472199 19044 raft_consensus.cc:1275] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Refusing update from remote peer 3da7ad8fa20a4591963b660e4c848e7f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:19:51.449299 26131 log.cc:927] Time spent T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: Append to log took a long time: real 0.053s	user 0.012s	sys 0.000s
I20260709 12:19:51.510789 26050 consensus_queue.cc:1048] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:19:51.523758 18338 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:51.544663 18182 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 57578.6us
W20260709 12:19:51.597149 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:51.966343 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.010852 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.040122 18339 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 53530.6us
W20260709 12:19:52.042210 18412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111) [suppressed 380 similar messages]
W20260709 12:19:52.094027 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.097568 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.104317 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.126015 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.172408 17157 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:52.217931 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.282948 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.321193 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 63774.5us
W20260709 12:19:52.335309 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.338953 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.412683 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 63187.1us
W20260709 12:19:52.425951 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.434588 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.546600 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.619738 19040 consensus_queue.cc:237] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:19:52.624567 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.641209 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.663470 18442 raft_consensus.cc:1275] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:19:52.668622 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:52.671101 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.672835 26098 consensus_queue.cc:1048] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:52.730116 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.735934 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.731586 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.738849 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.739667 26166 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 9 to 12, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
I20260709 12:19:52.744818 18442 raft_consensus.cc:2955] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 9 to 12, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:52.763134 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.765655 18338 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.786341 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:19:52.789290 18182 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 67a1783f49244bd5ba98721205d1ff6e with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
W20260709 12:19:52.805691 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.829161 18194 catalog_manager.cc:5697] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: config changed from index 9 to 12, VOTER 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) evicted. New cstate: current_term: 15 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } } }
W20260709 12:19:52.841851 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.846711 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:52.854450 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.855659 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.884104 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.887938 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:52.892375 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:19:52.946457 18357 tablet_service.cc:1558] Processing DeleteTablet for tablet 67a1783f49244bd5ba98721205d1ff6e with delete_type TABLET_DATA_TOMBSTONED (TS 3da7ad8fa20a4591963b660e4c848e7f not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:55214
W20260709 12:19:52.964407 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:52.969655 26175 tablet_replica.cc:333] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:19:52.985282 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:19:53.041889 26175 raft_consensus.cc:2243] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:53.043674 26175 raft_consensus.cc:2272] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:53.048698 26175 ts_tablet_manager.cc:1916] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:53.077414 26175 ts_tablet_manager.cc:1929] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:19:53.077967 26175 log.cc:1199] T 67a1783f49244bd5ba98721205d1ff6e P 3da7ad8fa20a4591963b660e4c848e7f: Deleting WAL directory at /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/wals/67a1783f49244bd5ba98721205d1ff6e
I20260709 12:19:53.080305 18185 catalog_manager.cc:5028] TS 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): tablet 67a1783f49244bd5ba98721205d1ff6e (table test-workload [id=8347973bc73f4ec692d005859baada6b]) successfully deleted
W20260709 12:19:53.149124 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:53.173092 17157 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:53.197283 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:53.290917 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:53.376754 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.405565 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:53.543051 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.602947 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.625708 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.631037 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:53.771224 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.836170 18338 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.855363 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:53.913183 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:54.030164 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:54.125062 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:54.173555 17157 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:54.336319 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.340166 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:54.611903 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.641462 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.642088 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.686542 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.743006 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.766865 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.793776 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.806205 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:54.876528 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:54.886293 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.058012 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.104640 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 46680.4us
W20260709 12:19:55.174100 17157 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:55.175336 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.224073 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:19:55.228585 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.231910 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.232549 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.238746 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.257058 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.284179 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.299122 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.333544 18338 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.353539 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.389788 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.401156 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.405822 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.504256 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.577824 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.600785 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.639689 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.762122 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.827474 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.850464 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:55.875288 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:55.946087 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.053706 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.057049 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.115123 19007 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:40227, user_credentials={real_user=slave}} blocked reactor thread for 90106.4us
W20260709 12:19:56.174768 17157 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:56.189606 18182 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 52177.2us
W20260709 12:19:56.192363 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:19:56.196688 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:56.210168 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.240417 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.245848 18338 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.252435 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:56.746901 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:56.888931 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:56.911159 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:56.927085 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:56.970652 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.049077 19006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111) [suppressed 452 similar messages]
W20260709 12:19:57.079478 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.109902 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.137260 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.174289 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.175262 17157 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:57.190938 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.397909 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.410030 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.494177 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.551260 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.556939 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.643842 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.648726 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.654106 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:19:57.684403 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.696002 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.718954 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.722967 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.806305 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.807175 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.811903 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:57.819116 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:57.987905 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.169595 18338 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.175743 17157 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:58.179666 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.202526 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.255003 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.275454 19009 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 106381us
W20260709 12:19:58.292058 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.294148 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.311463 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.434237 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.448098 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.449759 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.545292 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.639441 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.682869 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.715024 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.722767 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.763060 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:58.838801 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:58.896047 18338 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:59.154460 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.176334 17157 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 18321)
W20260709 12:19:59.278141 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:19:59.357131 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.425376 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:59.539070 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:59.547124 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.746898 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.781901 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.855695 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.904323 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.905098 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.920547 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.936969 18415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:44577, user_credentials={real_user=slave}} blocked reactor thread for 49725us
W20260709 12:19:59.954766 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:19:59.993433 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:19:59.996095 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.082732 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.118669 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.130862 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.146100 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.176802 17157 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 18321)
W20260709 12:20:00.181998 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:00.276614 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.317134 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.317847 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.384002 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:00.508899 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.515569 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.598351 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.675623 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.681986 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.695478 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.707305 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.731933 18338 consensus_peers.cc:597] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.769184 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.825362 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.838562 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:00.858078 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.889990 18338 consensus_peers.cc:597] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.977528 18338 consensus_peers.cc:597] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:00.984139 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.002035 18338 consensus_peers.cc:597] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:01.114840 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.125288 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.167315 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.388522 18338 consensus_peers.cc:597] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.415594 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.622169 18338 consensus_peers.cc:597] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:01.681484 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:01.740828 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:01.851653 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:01.855402 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:01.890597 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:01.928934 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:02.068512 18338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111) [suppressed 448 similar messages]
W20260709 12:20:02.220340 18338 consensus_peers.cc:597] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.230185 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.275766 18338 consensus_peers.cc:597] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.284272 18338 consensus_peers.cc:597] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:02.377604 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
W20260709 12:20:02.393098 18338 consensus_peers.cc:597] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.398422 18338 consensus_peers.cc:597] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.470180 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.479524 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:02.489351 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.514539 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:02.546842 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.559103 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.574458 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:02.639153 19006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 64541us
W20260709 12:20:02.682775 18338 consensus_peers.cc:597] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:02.702045 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:02.800603 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
W20260709 12:20:02.885125 19009 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.891525 19009 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.917094 19009 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.919858 18338 consensus_peers.cc:597] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.923822 19009 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.928232 18415 consensus_peers.cc:597] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.930150 18338 consensus_peers.cc:597] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:02.939386 18415 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.941301 18415 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.942755 18415 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.975198 19009 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.976007 19009 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:02.978825 19009 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.035825 18415 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.038549 18415 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.053965 19009 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.057332 18415 consensus_peers.cc:597] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.082278 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:03.082937 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.090096 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:03.093829 17157 tablet_replica.cc:333] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.095065 17157 raft_consensus.cc:2243] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.096812 17157 raft_consensus.cc:2272] T 7dd86332c91f4c8d8616b22b5daaf743 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.100505 17157 tablet_replica.cc:333] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.111923 17157 raft_consensus.cc:2243] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Raft consensus shutting down.
W20260709 12:20:03.113157 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.113420 17157 raft_consensus.cc:2272] T 83652a40670f4224a56a5a1347b9d1a1 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.116760 17157 tablet_replica.cc:333] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.117846 17157 raft_consensus.cc:2243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.120326 17157 raft_consensus.cc:2272] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.121033 18415 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:03.121799 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.125618 17157 tablet_replica.cc:333] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.132109 17157 raft_consensus.cc:2243] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.132948 17157 raft_consensus.cc:2272] T 0936d54d511a4cc3903264974a56ecb7 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.137283 17157 tablet_replica.cc:333] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.138201 17157 raft_consensus.cc:2243] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:20:03.143046 18338 consensus_peers.cc:597] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:03.145948 18415 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.144133 17157 raft_consensus.cc:2272] T 1d0efe89255942c4bd576ff4b96cd91d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.140789 19009 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.152400 17157 tablet_replica.cc:333] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.153347 17157 raft_consensus.cc:2243] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.156078 17157 raft_consensus.cc:2272] T 37ef0235d410491ab47198988075f6e4 P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.166981 17157 tablet_replica.cc:333] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.168032 17157 raft_consensus.cc:2243] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.169590 17157 raft_consensus.cc:2272] T 9fc8b80684174ecb8c933560e8143c49 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.172955 17157 tablet_replica.cc:333] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.176302 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.183470 17157 raft_consensus.cc:2243] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:03.183826 18415 consensus_peers.cc:597] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.186306 17157 raft_consensus.cc:2272] T 8903d70080d24236be955433a92791c7 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.197880 17157 tablet_replica.cc:333] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.200738 17157 raft_consensus.cc:2243] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:20:03.202181 18415 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.206202 17157 raft_consensus.cc:2272] T 9934c84956844d20a6258c5dca3833c6 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.209792 17157 tablet_replica.cc:333] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.210628 17157 raft_consensus.cc:2243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:03.222168 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.222365 17157 raft_consensus.cc:2272] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.223109 18338 consensus_peers.cc:597] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.225840 17157 tablet_replica.cc:333] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.226785 17157 raft_consensus.cc:2243] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.227747 17157 raft_consensus.cc:2272] T c749b52c1c8c44b78cee98b4b4d2ef29 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.230623 17157 tablet_replica.cc:333] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.231498 19009 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.245517 17157 raft_consensus.cc:2243] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.246762 17157 raft_consensus.cc:2272] T dc5742413fd242b88ca8c4668e6b8144 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.249603 17157 tablet_replica.cc:333] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.249536 18415 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.250846 17157 raft_consensus.cc:2243] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.251716 17157 raft_consensus.cc:2272] T 5d4d1763576e4c49849611360f37665a P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.259110 17157 tablet_replica.cc:333] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.259985 17157 raft_consensus.cc:2243] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.261312 17157 raft_consensus.cc:2272] T 977c0af9c8e94119b3d18a0ee7401458 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.265538 18415 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.268512 17157 tablet_replica.cc:333] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.269430 17157 raft_consensus.cc:2243] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.270884 17157 raft_consensus.cc:2272] T e98a95c8e3c14bcfa51b054f4829293b P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.272706 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.275063 17157 tablet_replica.cc:333] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.275703 18415 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.279821 17157 raft_consensus.cc:2243] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.280771 17157 raft_consensus.cc:2272] T b4688e522b8b48f7bb98e0832680f9e2 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.283766 17157 tablet_replica.cc:333] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.286723 17157 raft_consensus.cc:2243] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.287645 17157 raft_consensus.cc:2272] T 6021c985b4e44fe880350a321132a9b5 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.292243 17157 tablet_replica.cc:333] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.295790 17157 raft_consensus.cc:2243] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:03.300840 18415 consensus_peers.cc:597] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.301002 17157 raft_consensus.cc:2272] T 5461f19f144642beb790dcf7f1515a2d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.313194 17157 tablet_replica.cc:333] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.314174 17157 raft_consensus.cc:2243] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.315443 17157 raft_consensus.cc:2272] T 1a92f90eb0424e3d870764a2a328e508 P 3da7ad8fa20a4591963b660e4c848e7f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.318491 17157 tablet_replica.cc:333] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.322567 17157 raft_consensus.cc:2243] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.323318 17157 raft_consensus.cc:2272] T 01de3fbc59f243ba8a35edaa5fcfe86f P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.329306 17157 tablet_replica.cc:333] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.336128 17157 raft_consensus.cc:2243] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Raft consensus shutting down.
W20260709 12:20:03.337322 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.337461 17157 raft_consensus.cc:2272] T 2e4622d18b474b659ebea19e9aad0d05 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.340530 17157 tablet_replica.cc:333] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.344655 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.351905 17157 raft_consensus.cc:2243] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.352687 17157 raft_consensus.cc:2272] T c1f155085cb647ff957b468384227872 P 3da7ad8fa20a4591963b660e4c848e7f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.362636 18415 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:03.367772 17157 tablet_replica.cc:333] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.368695 17157 raft_consensus.cc:2243] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.369959 17157 raft_consensus.cc:2272] T 7d788f8182ca4b13bddd2acdf511873f P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.372766 17157 tablet_replica.cc:333] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.373651 17157 raft_consensus.cc:2243] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:20:03.374995 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.375017 17157 raft_consensus.cc:2272] T 57dbd4858e8f41a2af67d6de6eae9928 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.390326 17157 tablet_replica.cc:333] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.391136 17157 raft_consensus.cc:2243] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.391980 17157 raft_consensus.cc:2272] T b81b910b60b0422f9db4c0164608fcbb P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.394912 17157 tablet_replica.cc:333] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.395815 17157 raft_consensus.cc:2243] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.396634 17157 raft_consensus.cc:2272] T 80bc05a330c64471aaa556af75bea031 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.399753 17157 tablet_replica.cc:333] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.400545 17157 raft_consensus.cc:2243] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.403512 17157 raft_consensus.cc:2272] T 8e62e9fad92b449d9315da4aa77082b4 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.410569 17157 tablet_replica.cc:333] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.411504 17157 raft_consensus.cc:2243] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.412158 17157 raft_consensus.cc:2272] T 35973fcee55042b284c6f62e2681c304 P 3da7ad8fa20a4591963b660e4c848e7f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.418467 17157 tablet_replica.cc:333] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
W20260709 12:20:03.425242 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.427434 17157 raft_consensus.cc:2243] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.428805 17157 raft_consensus.cc:2272] T 6c5d2fad2696483cae18eab3ba337ec9 P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.431768 17157 tablet_replica.cc:333] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.432614 17157 raft_consensus.cc:2243] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.433998 17157 raft_consensus.cc:2272] T 2cbd14f2572940d797f0297b94365af4 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.436743 17157 tablet_replica.cc:333] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.451884 17157 raft_consensus.cc:2243] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.452708 17157 raft_consensus.cc:2272] T c5398d6474dd4eed946f45af9214825c P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.469234 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:03.472518 17157 tablet_replica.cc:333] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.473636 17157 raft_consensus.cc:2243] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.475582 17157 raft_consensus.cc:2272] T 72e8da9e5fd9449f93381781e125fada P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.478691 17157 tablet_replica.cc:333] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.479643 17157 raft_consensus.cc:2243] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.481158 17157 raft_consensus.cc:2272] T 85f32970b7374dcf9b9bac47791c71e3 P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.484191 17157 tablet_replica.cc:333] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.495952 17157 raft_consensus.cc:2243] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.497424 17157 raft_consensus.cc:2272] T adea31c273544d04b7f16f9b9a3ae610 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.498847 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.500677 17157 tablet_replica.cc:333] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.501714 17157 raft_consensus.cc:2243] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.502600 17157 raft_consensus.cc:2272] T 548b719daf794fb881c5606ebec2abb3 P 3da7ad8fa20a4591963b660e4c848e7f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.506167 17157 tablet_replica.cc:333] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.506996 17157 raft_consensus.cc:2243] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.509367 17157 raft_consensus.cc:2272] T d266f68528964b1981ffe1df49861b08 P 3da7ad8fa20a4591963b660e4c848e7f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.523128 17157 tablet_replica.cc:333] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.523962 17157 raft_consensus.cc:2243] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.524701 17157 raft_consensus.cc:2272] T 4b135a527cce4b14b789d23618d411b0 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.564266 17157 tablet_replica.cc:333] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.565349 17157 raft_consensus.cc:2243] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.566350 17157 raft_consensus.cc:2272] T e9decd1c7d7f475b8890a8519847dca5 P 3da7ad8fa20a4591963b660e4c848e7f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.577804 17157 tablet_replica.cc:333] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.589849 17157 raft_consensus.cc:2243] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.598937 17157 raft_consensus.cc:2272] T f53e315b86a94adb89240687e1a622ed P 3da7ad8fa20a4591963b660e4c848e7f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.602689 17157 tablet_replica.cc:333] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.619894 17157 raft_consensus.cc:2243] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.624754 17157 raft_consensus.cc:2272] T 6ab0a3f9b34149918a57aec5d20b4884 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.631467 17157 tablet_replica.cc:333] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.632323 17157 raft_consensus.cc:2243] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.633502 17157 raft_consensus.cc:2272] T 0931a54c049145dfad672173c0b29a40 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.641409 17157 tablet_replica.cc:333] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.642143 17157 raft_consensus.cc:2243] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:03.643462 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:03.645078 17157 raft_consensus.cc:2272] T 1686d6e6b8ae45aab8c5d9d916f8485d P 3da7ad8fa20a4591963b660e4c848e7f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.693045 17157 tablet_replica.cc:333] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.716521 17157 raft_consensus.cc:2243] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.727099 17157 raft_consensus.cc:2272] T 53c08deb8e86480a878b17c55d6bc089 P 3da7ad8fa20a4591963b660e4c848e7f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.738198 17157 tablet_replica.cc:333] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.740694 17157 raft_consensus.cc:2243] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:03.752714 17157 raft_consensus.cc:2272] T dc49a9d4c95c46f38ca4a279c913204d P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.766877 17157 tablet_replica.cc:333] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.767727 17157 raft_consensus.cc:2243] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.768536 17157 raft_consensus.cc:2272] T ca383fa1243547c3b32dfbd6857a3fba P 3da7ad8fa20a4591963b660e4c848e7f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:03.780856 17157 tablet_replica.cc:333] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f: stopping tablet replica
I20260709 12:20:03.781788 17157 raft_consensus.cc:2243] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:03.786120 17157 raft_consensus.cc:2272] T 353cbd244a354830a4e903a3b1be2b89 P 3da7ad8fa20a4591963b660e4c848e7f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:03.801496 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:03.983031 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:04.092403 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:04.179387 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.271399 26219 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.272008 26219 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.275884 26219 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.280405 19009 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.281553 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:04.282363 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.298357 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.299577 26222 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:04.301281 26211 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.302057 26211 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.314301 26211 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:20:04.316306 19040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
W20260709 12:20:04.322616 18415 leader_election.cc:336] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.323098 18415 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, 991dcd2a70c74fdb89ca9494cc09fb7b
I20260709 12:20:04.324611 26211 raft_consensus.cc:2749] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.338389 26219 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.339001 26219 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.342044 26219 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:04.347707 19009 leader_election.cc:336] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.421962 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:04.426137 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.444823 26225 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.445544 26225 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.487313 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
I20260709 12:20:04.488381 18446 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 19.
I20260709 12:20:04.505286 26225 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.514658 26222 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.522248 26222 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:04.522950 18412 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.524118 26230 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.524641 26230 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.526206 26222 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.526892 26230 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.540503 19009 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.541527 19009 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.545588 26231 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.546273 26231 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:04.549118 19006 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.549583 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.550472 26227 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:04.553408 19007 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:20:04.555158 19006 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.555724 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.562059 26230 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:04.562520 26230 raft_consensus.cc:493] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.562901 26230 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.564069 26231 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.568595 26231 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.572438 26230 raft_consensus.cc:515] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:04.581457 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.582453 26226 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.588511 26226 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.589846 26211 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.590358 26211 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.590777 26226 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.592351 26211 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.593127 26211 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.593665 26211 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.596041 26211 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.597882 18412 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.600600 18415 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.585398 26230 leader_election.cc:290] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:20:04.586674 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0efe89255942c4bd576ff4b96cd91d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3"
W20260709 12:20:04.587164 19009 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.609732 18412 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.610374 18446 raft_consensus.cc:3060] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.609809 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.611505 26211 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:04.614540 18412 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.610457 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.615976 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.617923 26230 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.624311 18446 raft_consensus.cc:2468] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 991dcd2a70c74fdb89ca9494cc09fb7b in term 20.
W20260709 12:20:04.625439 19009 leader_election.cc:336] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.626698 19007 leader_election.cc:304] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:20:04.627346 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.627527 26230 raft_consensus.cc:2804] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:04.629812 26230 raft_consensus.cc:697] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Becoming Leader. State: Replica: 991dcd2a70c74fdb89ca9494cc09fb7b, State: Running, Role: LEADER
I20260709 12:20:04.631049 26230 consensus_queue.cc:237] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.635113 26211 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.635787 26211 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:20:04.636379 18415 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.636888 18415 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.637580 26226 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.637878 26211 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.638505 18415 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.644990 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:20:04.642578 18412 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.646593 26211 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:04.647647 18415 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.648120 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.651504 26211 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:04.665282 18415 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.671147 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.671916 26211 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.684515 18195 catalog_manager.cc:5697] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b reported cstate change: term changed from 19 to 20, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68). New cstate: current_term: 20 leader_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:04.721071 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.745862 26211 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.746548 26211 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.748754 26211 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:04.746013 26230 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.749505 26230 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.751650 26230 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.752997 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.758553 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.759349 19009 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.759455 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.759763 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.759878 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.760921 26226 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:04.760921 26230 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.763630 26226 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.764189 26226 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.766556 26226 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.771903 18412 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.774087 26226 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.774642 26226 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:04.778359 26226 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.780977 18412 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.785964 18415 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.786504 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.787102 26230 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.787319 26226 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:04.787678 26230 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.789779 26230 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.797467 19009 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:04.800962 19006 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:04.801369 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:20:04.801482 18415 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.801932 18415 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:04.802534 26231 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.803045 26226 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.860301 26230 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:04.860823 26230 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:04.862962 26230 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:04.883409 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:04.884918 19009 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:04.885327 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:20:04.887920 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:04.888065 26230 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:04.911772 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.032073 17157 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18396)
I20260709 12:20:05.039595 26230 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:05.040210 26230 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:05.042546 26230 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:20:05.046669 18446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "177254018ba7448aa2b9110c9c9be9e3" is_pre_election: true
W20260709 12:20:05.057165 19009 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:05.057614 19009 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:05.059511 26231 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:05.068568 26228 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:05.069199 26228 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:05.071357 26228 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:20:05.073663 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" is_pre_election: true
I20260709 12:20:05.074330 19044 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 19.
I20260709 12:20:05.079720 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 
I20260709 12:20:05.080492 26228 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:05.080883 26228 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:05.081224 26228 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:05.083722 18415 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:05.090092 26228 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:05.098022 26228 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Requested vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 991dcd2a70c74fdb89ca9494cc09fb7b (127.16.193.68:43821)
I20260709 12:20:05.100455 19044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc49a9d4c95c46f38ca4a279c913204d" candidate_uuid: "177254018ba7448aa2b9110c9c9be9e3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b"
I20260709 12:20:05.101070 19044 raft_consensus.cc:3060] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:05.111902 18415 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:05.112736 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:05.118978 19044 raft_consensus.cc:2468] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177254018ba7448aa2b9110c9c9be9e3 in term 20.
I20260709 12:20:05.120378 18412 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3, 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:05.121454 26226 raft_consensus.cc:2804] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:05.122021 26226 raft_consensus.cc:697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Becoming Leader. State: Replica: 177254018ba7448aa2b9110c9c9be9e3, State: Running, Role: LEADER
I20260709 12:20:05.122963 26226 consensus_queue.cc:237] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:05.132200 18446 raft_consensus.cc:1275] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Refusing update from remote peer 991dcd2a70c74fdb89ca9494cc09fb7b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:05.133697 26230 consensus_queue.cc:1048] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:20:05.140688 19009 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:05.147588 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:05.154995 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:05.179545 26228 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:05.180064 26228 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:05.182168 26228 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:05.184554 18195 catalog_manager.cc:5697] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 reported cstate change: term changed from 19 to 20, leader changed from 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66) to 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67). New cstate: current_term: 20 leader_uuid: "177254018ba7448aa2b9110c9c9be9e3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:05.192878 18412 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:05.193802 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:05.194687 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:05.196476 26228 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:05.215093 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.317750 18415 consensus_peers.cc:597] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.338157 18415 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.364200 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.399646 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.420519 18415 consensus_peers.cc:597] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.440711 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003512a0 after lost signal to thread 17160
W20260709 12:20:05.441848 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd2e0 after lost signal to thread 18232
W20260709 12:20:05.443171 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800360ae0 after lost signal to thread 18473
W20260709 12:20:05.444594 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7fc0 after lost signal to thread 19068
W20260709 12:20:05.469579 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.476579 18415 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.481647 18415 consensus_peers.cc:597] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.481801 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.484721 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.488250 18415 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.528106 19009 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.531802 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.534333 19009 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.576916 19009 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.584208 18415 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.599532 18415 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.603425 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.626194 18415 consensus_peers.cc:597] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.657410 18415 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:05.672520 19044 raft_consensus.cc:1275] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Refusing update from remote peer 177254018ba7448aa2b9110c9c9be9e3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:20:05.674120 19009 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:05.675679 26211 consensus_queue.cc:1048] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:20:05.679930 19009 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.688273 19009 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.703620 18415 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.742394 18415 consensus_peers.cc:597] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:05.745234 19009 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.781103 18415 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.798890 18415 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.803920 18415 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.806713 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.810567 19009 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.815886 19009 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.851105 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.863354 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.868906 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:05.870460 19009 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.915941 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:05.919718 18415 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:05.950970 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:06.026294 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:06.032590 17157 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:06.055912 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:06.147818 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:06.186177 18415 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:06.251840 26228 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.252390 26228 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.254531 26228 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:06.262490 26228 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.262971 26228 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:20:06.264672 18415 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.265095 26228 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.266530 18412 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.269389 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.274302 26228 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:06.275300 18415 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.276350 18412 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.276762 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.279680 26228 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.320745 26228 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.321308 26228 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.323585 26228 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.328272 18412 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:06.329186 18415 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.329638 18415 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.330863 26228 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:06.353855 26230 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.354425 26230 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:06.356796 26230 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.363554 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.364445 26230 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.364952 26230 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:06.367188 26230 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.373035 19009 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.374907 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.375384 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.376232 26230 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:06.383108 19009 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.383591 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.389138 26230 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:06.399140 26211 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.399787 26211 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.401959 26211 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.426575 18412 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:06.446620 18415 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.451608 26245 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.453454 26245 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:06.455714 26245 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.463701 19006 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.464592 18415 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.478633 26211 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.486642 26211 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.487175 26211 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.489002 26228 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.489234 26211 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:06.489622 26228 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.490303 26226 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.490869 26226 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.491897 26228 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:06.493042 26226 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.532594 19009 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.536788 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.548569 26230 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:06.553109 18415 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.571966 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.572628 18415 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.583442 18412 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.584506 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.588642 26211 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:06.590270 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.591665 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.594465 26211 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:06.595654 18412 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.596131 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.596872 26211 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:06.611164 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:06.611756 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:06.621069 26245 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.621671 26245 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:06.623984 26245 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.638128 19006 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:06.643113 19009 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.643716 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.644438 26246 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:06.717783 26211 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.718361 26211 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.722519 26211 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.731019 18412 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:06.731875 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:06.732313 18415 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.733534 26211 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.734525 26245 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.735127 26245 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:06.759083 19009 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.764110 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.764547 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.766712 26246 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.770290 26245 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.786929 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:06.822871 26245 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.823529 26245 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:06.825767 26245 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.829815 19009 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.837087 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.842010 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.843854 26246 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:06.869191 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:06.892146 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:06.917098 26211 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:06.917735 26211 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:06.921370 26211 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:06.932157 18415 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:06.980795 18412 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:06.982736 18412 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:06.984834 26226 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:06.985940 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:07.019537 26245 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:07.020102 26245 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:07.026930 19009 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:07.027776 26245 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:07.033068 17157 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:07.039333 19006 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:07.051971 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:07.054096 26245 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:07.069168 18415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111) [suppressed 642 similar messages]
W20260709 12:20:07.149822 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.216708 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.296743 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.369560 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:07.380600 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.551528 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.664040 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.695029 18415 consensus_peers.cc:597] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.705663 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:07.708570 19009 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:07.818401 18415 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.913829 19009 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.935858 18415 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.939239 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.940186 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:20:07.949486 26247 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:07.950167 26247 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:20:07.958185 19009 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:07.960575 26247 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:07.963564 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:07.963631 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.965245 19009 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.965556 18415 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.973080 18415 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:07.973711 18415 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.976961 19009 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:07.978935 18412 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:07.979542 18412 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:07.980798 26247 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:08.001070 18415 consensus_peers.cc:597] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.005733 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:08.008735 18415 consensus_peers.cc:597] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.016396 18415 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.034205 17157 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:08.050290 19009 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.089906 18415 consensus_peers.cc:597] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.104586 19009 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:08.113929 26226 raft_consensus.cc:493] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.114578 26226 raft_consensus.cc:515] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:08.118763 26226 leader_election.cc:290] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.123056 18415 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:08.162492 19009 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.168726 18415 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.218888 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:08.227548 18415 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.242679 18415 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.248363 19009 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.289712 19009 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.310277 18415 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.312536 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:08.372550 26226 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.373169 26226 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:08.377616 26226 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.395217 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:08.439325 19009 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:08.447044 26245 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.447753 26245 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:08.457172 26245 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.467499 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:08.478266 19009 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:08.488409 18415 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:08.502544 18415 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:08.506866 26211 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.507445 26211 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:08.509548 26211 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.511003 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:08.511538 19006 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:08.514330 26246 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:08.519279 26211 raft_consensus.cc:493] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.519847 26211 raft_consensus.cc:515] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:08.522050 26211 leader_election.cc:290] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.523105 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:08.537585 18415 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:08.557415 26247 raft_consensus.cc:493] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.558007 26247 raft_consensus.cc:515] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:08.560266 26247 leader_election.cc:290] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:08.585152 26245 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.585677 26245 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:08.587858 26245 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.594699 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:08.604619 18415 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:08.618199 19009 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:08.625547 19006 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:08.626035 19006 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:08.639339 26246 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:08.645660 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:08.656257 18415 consensus_peers.cc:597] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:08.656777 18415 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:08.856349 26246 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:08.856886 26246 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:08.859058 26246 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:08.864562 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:08.865363 19009 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:08.865823 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:08.866715 26246 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:08.871775 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:08.926414 18415 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:09.035279 17157 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:09.069710 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:09.073037 18412 leader_election.cc:336] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.081238 18412 leader_election.cc:304] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.091403 26211 raft_consensus.cc:2749] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:09.105069 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.105623 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.106568 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.107005 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.108467 26211 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:09.109575 18412 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.111603 26211 raft_consensus.cc:493] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.112130 26211 raft_consensus.cc:515] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:09.114015 26211 leader_election.cc:290] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.121177 18415 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:09.115717 18412 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.136034 26247 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.136585 26247 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:09.138654 26247 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.157586 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:09.162375 26211 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:09.163475 18412 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:37771, user_credentials={real_user=slave}} blocked reactor thread for 53915us
W20260709 12:20:09.165025 18412 leader_election.cc:336] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.165457 18412 leader_election.cc:304] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.171491 26211 raft_consensus.cc:2749] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:09.173626 18412 leader_election.cc:336] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.174105 18412 leader_election.cc:304] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.174854 26211 raft_consensus.cc:2749] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:09.242251 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.242805 18412 leader_election.cc:336] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.243219 18412 leader_election.cc:304] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.251905 26247 raft_consensus.cc:2749] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:09.253733 18412 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.259789 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.260479 26211 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:09.275914 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:09.420743 26245 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.421258 26245 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:09.423596 26245 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.428632 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:09.432039 19009 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:09.432530 19009 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.434340 26245 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.443650 26245 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.444159 26245 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:09.452119 19009 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:09.452809 26245 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.455106 19006 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.455605 19006 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.456820 26245 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:09.506848 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.709370 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:09.731719 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:09.768210 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.788014 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:09.798657 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:09.811765 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:09.815572 26253 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.816175 26253 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:09.818261 26253 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.826037 19009 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:09.826785 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:09.827154 19006 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:09.827913 26253 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:09.849450 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:09.860553 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:09.917616 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:09.990712 26211 raft_consensus.cc:493] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:09.991206 26211 raft_consensus.cc:515] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:09.993188 26211 leader_election.cc:290] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:09.998744 18412 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:10.035737 17157 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:10.091969 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.095151 18412 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:10.215583 26246 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:10.216120 26246 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:10.218308 26246 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:10.235638 19009 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:10.248924 18412 consensus_peers.cc:597] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:10.253177 19006 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:10.253778 19006 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:10.255254 26246 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:10.258553 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:10.325605 19009 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.347997 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:10.350700 19009 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:10.404032 19009 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.448015 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:20:10.460862 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.526690 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:10.558343 19009 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.565187 19009 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.568079 19009 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.616482 19009 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.670419 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:10.689188 18415 leader_election.cc:336] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:10.689822 18415 leader_election.cc:304] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:10.691835 26254 raft_consensus.cc:2749] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:10.701304 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:10.707170 18412 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:10.709093 26254 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:10.709589 26254 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:10.711760 26254 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:10.719537 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:10.722496 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:10.722962 18412 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:10.723685 26254 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:10.737411 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:10.741904 18412 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:10.764346 26254 raft_consensus.cc:493] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:10.764884 26254 raft_consensus.cc:515] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:10.767110 26254 leader_election.cc:290] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:10.772666 18412 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:10.774250 18415 leader_election.cc:336] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:10.774633 18415 leader_election.cc:304] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:10.783375 26211 raft_consensus.cc:2749] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:10.789819 18415 consensus_peers.cc:597] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.792469 19009 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.814821 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:10.819398 18415 consensus_peers.cc:597] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.831861 19009 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.839084 18415 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.845609 18415 consensus_peers.cc:597] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:10.917757 19009 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.113859 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:11.188109 17157 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 18396)
W20260709 12:20:11.198448 18415 consensus_peers.cc:597] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.198949 18415 consensus_peers.cc:597] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.199801 18415 consensus_peers.cc:597] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.200448 18415 consensus_peers.cc:597] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.200888 18415 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.224174 19009 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.323828 18412 consensus_peers.cc:597] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:11.349229 18415 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.355857 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:11.382180 18412 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:11.453231 26255 raft_consensus.cc:493] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:11.453842 26255 raft_consensus.cc:515] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:11.455891 26255 leader_election.cc:290] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:11.460038 18415 consensus_peers.cc:597] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.463860 19009 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:11.464675 19006 leader_election.cc:336] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:11.465085 19006 leader_election.cc:304] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:11.466216 26255 raft_consensus.cc:2749] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.528664 26260 raft_consensus.cc:493] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:11.529340 26260 raft_consensus.cc:515] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:11.535983 26260 leader_election.cc:290] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:11.537573 18415 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:11.538789 18412 leader_election.cc:336] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:11.546581 18412 leader_election.cc:304] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:11.555158 26260 raft_consensus.cc:2749] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:11.561640 18415 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.564855 18415 consensus_peers.cc:597] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.583609 18415 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.596560 18412 consensus_peers.cc:597] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:11.606614 18412 consensus_peers.cc:597] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:11.630589 18415 consensus_peers.cc:597] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:11.858490 18415 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.893214 18415 consensus_peers.cc:597] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:11.960489 18412 consensus_peers.cc:597] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:11.990551 26259 raft_consensus.cc:493] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:11.991119 26259 raft_consensus.cc:515] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
I20260709 12:20:11.994175 26259 leader_election.cc:290] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:11.998762 18412 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:11.999322 18415 leader_election.cc:336] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:11.999743 18415 leader_election.cc:304] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:12.000423 26259 raft_consensus.cc:2749] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:12.071151 19006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111) [suppressed 587 similar messages]
W20260709 12:20:12.135838 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800106160 after lost signal to thread 17160
W20260709 12:20:12.136965 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800164020 after lost signal to thread 18232
W20260709 12:20:12.138243 19065 debug-util.cc:398] Leaking SignalData structure 0x7b08001ce860 after lost signal to thread 18473
W20260709 12:20:12.139422 19065 debug-util.cc:398] Leaking SignalData structure 0x7b0800130ac0 after lost signal to thread 19068
I20260709 12:20:12.239316 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:20:12.267638 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:12.268304 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:12.268922 18412 consensus_peers.cc:597] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:20:12.372440 26256 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:12.373147 26256 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:12.381783 26256 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
I20260709 12:20:12.386456 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
W20260709 12:20:12.389900 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:12.391767 18412 consensus_peers.cc:597] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:12.395550 26260 raft_consensus.cc:493] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:12.398504 26260 raft_consensus.cc:515] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } }
W20260709 12:20:12.405746 19009 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:12.406165 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:20:12.407797 18412 consensus_peers.cc:597] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:20:12.407955 26261 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.419421 26260 leader_election.cc:290] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:12.419710 18412 consensus_peers.cc:597] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:12.407497 26246 raft_consensus.cc:493] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:12.422065 26246 raft_consensus.cc:515] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:12.422606 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:12.424641 26246 leader_election.cc:290] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:12.435150 18412 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:12.443351 18415 leader_election.cc:336] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:12.443783 18415 leader_election.cc:304] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 177254018ba7448aa2b9110c9c9be9e3; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
W20260709 12:20:12.443827 19006 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:12.444533 26264 raft_consensus.cc:2749] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:20:12.446362 19009 leader_election.cc:336] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:12.446851 19009 leader_election.cc:304] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:12.447558 26246 raft_consensus.cc:2749] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:12.457218 18412 consensus_peers.cc:597] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:12.461520 19007 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.474308 18412 consensus_peers.cc:597] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:12.492825 19007 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.497349 19007 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.508342 19007 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:12.526233 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:12.527268 17157 tablet_replica.cc:333] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.528275 17157 raft_consensus.cc:2243] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.529273 17157 raft_consensus.cc:2272] T 83652a40670f4224a56a5a1347b9d1a1 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.532282 17157 tablet_replica.cc:333] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.533134 17157 raft_consensus.cc:2243] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.534143 17157 raft_consensus.cc:2272] T 1d0efe89255942c4bd576ff4b96cd91d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.536780 17157 tablet_replica.cc:333] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.537650 17157 raft_consensus.cc:2243] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.538273 17157 raft_consensus.cc:2272] T 9e2ee63a449648bfb064fadee177cadb P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.540964 17157 tablet_replica.cc:333] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.541869 17157 raft_consensus.cc:2243] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.543007 17157 raft_consensus.cc:2272] T 37ef0235d410491ab47198988075f6e4 P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.545568 17157 tablet_replica.cc:333] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.546380 17157 raft_consensus.cc:2243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.547446 17157 raft_consensus.cc:2272] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.550104 17157 tablet_replica.cc:333] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.550941 17157 raft_consensus.cc:2243] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.551529 17157 raft_consensus.cc:2272] T 9934c84956844d20a6258c5dca3833c6 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.554280 17157 tablet_replica.cc:333] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.555151 17157 raft_consensus.cc:2243] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.555866 17157 raft_consensus.cc:2272] T 9fc8b80684174ecb8c933560e8143c49 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.558485 17157 tablet_replica.cc:333] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.559430 17157 raft_consensus.cc:2243] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.560951 17157 raft_consensus.cc:2272] T dc5742413fd242b88ca8c4668e6b8144 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.563519 17157 tablet_replica.cc:333] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.564254 17157 raft_consensus.cc:2243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.565032 17157 raft_consensus.cc:2272] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.567618 17157 tablet_replica.cc:333] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.568353 17157 raft_consensus.cc:2243] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.569344 17157 raft_consensus.cc:2272] T 0931a54c049145dfad672173c0b29a40 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.571928 17157 tablet_replica.cc:333] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.572659 17157 raft_consensus.cc:2243] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.573625 17157 raft_consensus.cc:2272] T b4688e522b8b48f7bb98e0832680f9e2 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.576035 17157 tablet_replica.cc:333] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.576690 17157 raft_consensus.cc:2243] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.577514 17157 raft_consensus.cc:2272] T 6021c985b4e44fe880350a321132a9b5 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.579437 17157 tablet_replica.cc:333] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.579967 17157 raft_consensus.cc:2243] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.581034 17157 raft_consensus.cc:2272] T 5461f19f144642beb790dcf7f1515a2d P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.583272 17157 tablet_replica.cc:333] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
W20260709 12:20:12.583518 19006 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:20:12.583979 17157 raft_consensus.cc:2243] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.584488 17157 raft_consensus.cc:2272] T 977c0af9c8e94119b3d18a0ee7401458 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.586333 17157 tablet_replica.cc:333] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.586874 17157 raft_consensus.cc:2243] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.588153 17157 raft_consensus.cc:2272] T d8bf6d66a261490caa69a80b24ea015f P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.589923 17157 tablet_replica.cc:333] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.590457 17157 raft_consensus.cc:2243] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.590916 17157 raft_consensus.cc:2272] T 1a92f90eb0424e3d870764a2a328e508 P 177254018ba7448aa2b9110c9c9be9e3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.592798 17157 tablet_replica.cc:333] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.593320 17157 raft_consensus.cc:2243] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.594117 17157 raft_consensus.cc:2272] T fafcfc4614304a81b0eadc8f445bfdb8 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.596086 17157 tablet_replica.cc:333] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.596622 17157 raft_consensus.cc:2243] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.597397 17157 raft_consensus.cc:2272] T f53e315b86a94adb89240687e1a622ed P 177254018ba7448aa2b9110c9c9be9e3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.599495 17157 tablet_replica.cc:333] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.600255 17157 raft_consensus.cc:2243] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.601696 17157 raft_consensus.cc:2272] T c1f155085cb647ff957b468384227872 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.604486 17157 tablet_replica.cc:333] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.605214 17157 raft_consensus.cc:2243] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.605957 17157 raft_consensus.cc:2272] T 67a1783f49244bd5ba98721205d1ff6e P 177254018ba7448aa2b9110c9c9be9e3 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.608191 17157 tablet_replica.cc:333] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.608894 17157 raft_consensus.cc:2243] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.609925 17157 raft_consensus.cc:2272] T 01de3fbc59f243ba8a35edaa5fcfe86f P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.612361 17157 tablet_replica.cc:333] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.612869 17157 raft_consensus.cc:2243] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.613301 17157 raft_consensus.cc:2272] T a971a418238d45d288b49960c0abeea0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.615072 17157 tablet_replica.cc:333] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.615641 17157 raft_consensus.cc:2243] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.616602 17157 raft_consensus.cc:2272] T b81b910b60b0422f9db4c0164608fcbb P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.618460 17157 tablet_replica.cc:333] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.619000 17157 raft_consensus.cc:2243] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.619517 17157 raft_consensus.cc:2272] T 53c08deb8e86480a878b17c55d6bc089 P 177254018ba7448aa2b9110c9c9be9e3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.621829 17157 tablet_replica.cc:333] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.622545 17157 raft_consensus.cc:2243] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.623363 17157 raft_consensus.cc:2272] T 85f32970b7374dcf9b9bac47791c71e3 P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.625679 17157 tablet_replica.cc:333] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.626458 17157 raft_consensus.cc:2243] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:20:12.627569 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:12.627946 17157 raft_consensus.cc:2272] T 72e8da9e5fd9449f93381781e125fada P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.629976 17157 tablet_replica.cc:333] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.630511 17157 raft_consensus.cc:2243] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.631567 17157 raft_consensus.cc:2272] T 6c5d2fad2696483cae18eab3ba337ec9 P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.633457 17157 tablet_replica.cc:333] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.633978 17157 raft_consensus.cc:2243] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.634443 17157 raft_consensus.cc:2272] T 389aa517d744435bb043ff8c52d86c2d P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.636287 17157 tablet_replica.cc:333] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.636816 17157 raft_consensus.cc:2243] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.637243 17157 raft_consensus.cc:2272] T 35973fcee55042b284c6f62e2681c304 P 177254018ba7448aa2b9110c9c9be9e3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.639338 17157 tablet_replica.cc:333] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.639982 17157 raft_consensus.cc:2243] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.640568 17157 raft_consensus.cc:2272] T 2cbd14f2572940d797f0297b94365af4 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:12.641808 19007 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:12.642802 17157 tablet_replica.cc:333] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.643544 17157 raft_consensus.cc:2243] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.644153 17157 raft_consensus.cc:2272] T c5398d6474dd4eed946f45af9214825c P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.646449 17157 tablet_replica.cc:333] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.647137 17157 raft_consensus.cc:2243] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.648102 17157 raft_consensus.cc:2272] T 3153d259c2254b2d9bdc71d8322cafc0 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.649986 17157 tablet_replica.cc:333] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.650501 17157 raft_consensus.cc:2243] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.650949 17157 raft_consensus.cc:2272] T 7d788f8182ca4b13bddd2acdf511873f P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.652717 17157 tablet_replica.cc:333] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.653250 17157 raft_consensus.cc:2243] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.654001 17157 raft_consensus.cc:2272] T ca383fa1243547c3b32dfbd6857a3fba P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.656051 17157 tablet_replica.cc:333] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.656715 17157 raft_consensus.cc:2243] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.657312 17157 raft_consensus.cc:2272] T 0071c33745ae42e48eca14d86aa5fac6 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.659708 17157 tablet_replica.cc:333] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.660197 17157 raft_consensus.cc:2243] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.661067 17157 raft_consensus.cc:2272] T e7d0369b4aea4afb960bf859e9b7738c P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.662997 17157 tablet_replica.cc:333] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.663534 17157 raft_consensus.cc:2243] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.663959 17157 raft_consensus.cc:2272] T 353cbd244a354830a4e903a3b1be2b89 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.665664 17157 tablet_replica.cc:333] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.666143 17157 raft_consensus.cc:2243] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.667085 17157 raft_consensus.cc:2272] T dc49a9d4c95c46f38ca4a279c913204d P 177254018ba7448aa2b9110c9c9be9e3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.669310 17157 tablet_replica.cc:333] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.670092 17157 raft_consensus.cc:2243] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.670696 17157 raft_consensus.cc:2272] T cc5f8db079f74a52b906175ad2c4cb56 P 177254018ba7448aa2b9110c9c9be9e3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:12.671214 19006 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:12.671267 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08000a76e0 after lost signal to thread 17160
W20260709 12:20:12.672593 18229 debug-util.cc:398] Leaking SignalData structure 0x7b080038e6c0 after lost signal to thread 18232
I20260709 12:20:12.672950 17157 tablet_replica.cc:333] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.673493 17157 raft_consensus.cc:2243] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:20:12.673868 18229 debug-util.cc:398] Leaking SignalData structure 0x7b0800064940 after lost signal to thread 18473
I20260709 12:20:12.674460 17157 raft_consensus.cc:2272] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:12.675107 18229 debug-util.cc:398] Leaking SignalData structure 0x7b08003a3300 after lost signal to thread 19068
I20260709 12:20:12.676770 17157 tablet_replica.cc:333] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.677371 17157 raft_consensus.cc:2243] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.677981 17157 raft_consensus.cc:2272] T b9f2e3c205c14e43b532c52bc7fa4c90 P 177254018ba7448aa2b9110c9c9be9e3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.680060 17157 tablet_replica.cc:333] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.680779 17157 raft_consensus.cc:2243] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.681349 17157 raft_consensus.cc:2272] T 4b135a527cce4b14b789d23618d411b0 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.683768 17157 tablet_replica.cc:333] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.684442 17157 raft_consensus.cc:2243] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.685482 17157 raft_consensus.cc:2272] T d266f68528964b1981ffe1df49861b08 P 177254018ba7448aa2b9110c9c9be9e3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.687887 17157 tablet_replica.cc:333] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.688416 17157 raft_consensus.cc:2243] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.689254 17157 raft_consensus.cc:2272] T 1686d6e6b8ae45aab8c5d9d916f8485d P 177254018ba7448aa2b9110c9c9be9e3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.691182 17157 tablet_replica.cc:333] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.691684 17157 raft_consensus.cc:2243] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:12.692278 17157 raft_consensus.cc:2272] T ad852e5220a14f4898d2f069a5fb4a63 P 177254018ba7448aa2b9110c9c9be9e3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:12.695371 17157 tablet_replica.cc:333] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3: stopping tablet replica
I20260709 12:20:12.696105 17157 raft_consensus.cc:2243] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:12.697537 17157 raft_consensus.cc:2272] T e9decd1c7d7f475b8890a8519847dca5 P 177254018ba7448aa2b9110c9c9be9e3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:12.702234 19007 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.708833 19007 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.706875 19009 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:12.768019 19007 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.770885 19009 consensus_peers.cc:597] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:12.779466 19007 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.800657 19009 consensus_peers.cc:597] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:12.799191 19007 consensus_peers.cc:597] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.812121 19007 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.823518 19007 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.837445 19007 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.877147 19006 consensus_peers.cc:597] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:12.877615 19007 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.905531 19007 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:12.910846 18470 debug-util.cc:398] Leaking SignalData structure 0x7b0800383700 after lost signal to thread 17160
W20260709 12:20:12.913080 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1120 after lost signal to thread 18232
W20260709 12:20:12.915505 18470 debug-util.cc:398] Leaking SignalData structure 0x7b0800380ec0 after lost signal to thread 18473
W20260709 12:20:12.917996 18470 debug-util.cc:398] Leaking SignalData structure 0x7b08000ae960 after lost signal to thread 19068
W20260709 12:20:12.937857 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:20:12.966915 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:12.983762 19006 consensus_peers.cc:597] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:13.058535 19006 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:13.091794 19009 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.134464 19009 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:13.139401 26246 raft_consensus.cc:493] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:13.140400 26246 raft_consensus.cc:515] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:13.147281 19006 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:13.148798 26246 leader_election.cc:290] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:13.172019 19009 leader_election.cc:336] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.172490 19009 leader_election.cc:304] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:13.174005 26261 raft_consensus.cc:2749] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:13.220064 19006 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:13.222875 19009 consensus_peers.cc:597] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:13.262938 26246 raft_consensus.cc:493] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:13.263698 26246 raft_consensus.cc:515] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.265847 26246 leader_election.cc:290] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:13.306741 19006 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:13.308184 19009 leader_election.cc:336] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.308640 19009 leader_election.cc:304] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:13.310824 26256 raft_consensus.cc:2749] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:13.338886 19009 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.341701 19009 consensus_peers.cc:597] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.442714 19009 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.443267 19009 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.470058 19006 consensus_peers.cc:597] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:20:13.470600 19006 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:13.473305 19009 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:13.487370 19006 consensus_peers.cc:597] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:20:13.611557 19006 consensus_peers.cc:597] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:13.669332 26246 raft_consensus.cc:493] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.670037 26246 raft_consensus.cc:515] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.672930 26246 leader_election.cc:290] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.676990 19009 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.693782 26246 raft_consensus.cc:493] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.694497 26246 raft_consensus.cc:515] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.696846 26246 leader_election.cc:290] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.699457 19007 leader_election.cc:336] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:13.699942 19007 leader_election.cc:304] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:13.701655 26246 raft_consensus.cc:2749] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:13.705418 19006 leader_election.cc:336] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:13.713090 19007 leader_election.cc:336] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:13.714676 19007 leader_election.cc:304] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:13.716683 26246 raft_consensus.cc:2749] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:13.802333 26246 raft_consensus.cc:493] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.803021 26246 raft_consensus.cc:515] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.806340 26246 leader_election.cc:290] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.811146 19009 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.815465 26246 raft_consensus.cc:493] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.816213 26246 raft_consensus.cc:515] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.819110 26246 leader_election.cc:290] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.829360 19009 leader_election.cc:336] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.833878 26246 raft_consensus.cc:493] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.837003 26246 raft_consensus.cc:515] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:13.843586 19007 leader_election.cc:336] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:13.844100 19007 leader_election.cc:304] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
W20260709 12:20:13.849753 19006 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:13.853511 26246 leader_election.cc:290] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:20:13.854092 26256 raft_consensus.cc:2749] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:13.858047 19009 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.858480 19009 leader_election.cc:304] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:13.859417 26261 raft_consensus.cc:2749] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.862746 26256 raft_consensus.cc:493] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.863418 26256 raft_consensus.cc:515] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.865929 26256 leader_election.cc:290] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:20:13.889758 26256 raft_consensus.cc:493] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:13.890457 26256 raft_consensus.cc:515] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.893750 26256 leader_election.cc:290] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:13.897303 19007 leader_election.cc:336] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
W20260709 12:20:13.904776 19006 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:13.910908 26256 raft_consensus.cc:493] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.913947 26256 raft_consensus.cc:515] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.911587 19007 leader_election.cc:304] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:13.918097 26261 raft_consensus.cc:2749] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:13.915131 19009 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.932336 26261 raft_consensus.cc:493] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.933211 26261 raft_consensus.cc:515] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:13.934281 17157 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18470)
I20260709 12:20:13.935945 26261 leader_election.cc:290] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.942565 19006 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:13.943912 26256 leader_election.cc:290] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.945873 19009 leader_election.cc:336] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.946381 19009 leader_election.cc:304] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:13.947400 26256 raft_consensus.cc:2749] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.947573 26266 raft_consensus.cc:493] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.948453 26266 raft_consensus.cc:515] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.951128 26266 leader_election.cc:290] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
I20260709 12:20:13.952214 26266 raft_consensus.cc:493] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.952977 26266 raft_consensus.cc:515] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.957116 26266 leader_election.cc:290] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.962313 19006 leader_election.cc:336] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:13.963379 19006 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:13.974395 19009 leader_election.cc:336] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:13.978544 26266 raft_consensus.cc:493] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:13.979437 26266 raft_consensus.cc:515] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:13.982514 26266 leader_election.cc:290] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:13.985595 19007 leader_election.cc:336] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
W20260709 12:20:13.986641 19007 leader_election.cc:336] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:13.987094 19007 leader_election.cc:304] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:13.990079 26267 raft_consensus.cc:2749] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:13.999145 19009 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:14.013588 26267 raft_consensus.cc:493] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:14.014667 26267 raft_consensus.cc:515] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:14.017414 26267 leader_election.cc:290] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:14.027447 19007 leader_election.cc:336] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.029933 19007 leader_election.cc:304] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:14.031523 26267 raft_consensus.cc:2749] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:14.029397 19009 leader_election.cc:336] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
W20260709 12:20:14.031217 19007 leader_election.cc:336] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.033520 19007 leader_election.cc:304] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.034461 26267 raft_consensus.cc:2749] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:14.035725 19007 leader_election.cc:336] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.036245 19007 leader_election.cc:304] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.037108 26267 raft_consensus.cc:2749] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:14.038471 19007 leader_election.cc:336] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.038977 19007 leader_election.cc:304] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.039877 26267 raft_consensus.cc:2749] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.041399 26267 raft_consensus.cc:493] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 177254018ba7448aa2b9110c9c9be9e3)
I20260709 12:20:14.042141 26267 raft_consensus.cc:515] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
W20260709 12:20:14.049787 19006 leader_election.cc:336] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:14.052311 26267 leader_election.cc:290] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227)
W20260709 12:20:14.052912 19007 leader_election.cc:336] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.053706 19007 leader_election.cc:304] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:14.055895 26267 raft_consensus.cc:2749] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:14.085805 19007 leader_election.cc:336] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.086265 19007 leader_election.cc:304] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, 3da7ad8fa20a4591963b660e4c848e7f
I20260709 12:20:14.087021 26267 raft_consensus.cc:2749] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:14.090284 19007 leader_election.cc:336] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111)
I20260709 12:20:14.090804 19007 leader_election.cc:304] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 177254018ba7448aa2b9110c9c9be9e3, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.091660 26267 raft_consensus.cc:2749] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.137526 26267 raft_consensus.cc:493] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:14.138314 26267 raft_consensus.cc:515] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:14.147312 26267 leader_election.cc:290] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:14.148823 19006 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:14.151211 19009 leader_election.cc:336] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:14.151639 19009 leader_election.cc:304] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.152421 26267 raft_consensus.cc:2749] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:14.692351 19007 consensus_peers.cc:597] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:14.750875 26267 raft_consensus.cc:493] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3da7ad8fa20a4591963b660e4c848e7f)
I20260709 12:20:14.751579 26267 raft_consensus.cc:515] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: false } }
I20260709 12:20:14.753839 26267 leader_election.cc:290] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771), 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577)
W20260709 12:20:14.758821 19006 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
W20260709 12:20:14.759390 19009 leader_election.cc:336] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111)
I20260709 12:20:14.759884 19009 leader_election.cc:304] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 991dcd2a70c74fdb89ca9494cc09fb7b; no voters: 3da7ad8fa20a4591963b660e4c848e7f, e23c0ce8d89940da8eac6a6002e349c5
I20260709 12:20:14.760565 26267 raft_consensus.cc:2749] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:14.792320 19009 consensus_peers.cc:597] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:14.798748 19006 consensus_peers.cc:597] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:20:14.856326 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
W20260709 12:20:14.858592 19006 consensus_peers.cc:597] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:20:14.889053 19007 consensus_peers.cc:597] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:14.905793 19007 consensus_peers.cc:597] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:14.960831 19007 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:14.990113 17157 tablet_server.cc:179] TabletServer@127.16.193.68:0 shutting down...
W20260709 12:20:15.015434 19007 consensus_peers.cc:597] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:15.039955 19006 consensus_peers.cc:597] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:20:15.055040 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:15.056057 17157 tablet_replica.cc:333] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.056989 17157 raft_consensus.cc:2243] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.057878 17157 raft_consensus.cc:2272] T 72e8da9e5fd9449f93381781e125fada P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.060415 17157 tablet_replica.cc:333] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.061380 17157 raft_consensus.cc:2243] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.062831 17157 raft_consensus.cc:2272] T 8903d70080d24236be955433a92791c7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.066804 17157 tablet_replica.cc:333] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.067644 17157 raft_consensus.cc:2243] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 LEADER]: Raft consensus shutting down.
W20260709 12:20:15.067907 19006 consensus_peers.cc:597] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): Couldn't send request to peer e23c0ce8d89940da8eac6a6002e349c5. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:20:15.071645 17157 raft_consensus.cc:2272] T 67a1783f49244bd5ba98721205d1ff6e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.074421 17157 tablet_replica.cc:333] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.075318 17157 raft_consensus.cc:2243] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.077239 17157 raft_consensus.cc:2272] T 1d0efe89255942c4bd576ff4b96cd91d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.081249 17157 tablet_replica.cc:333] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.082098 17157 raft_consensus.cc:2243] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.083019 17157 raft_consensus.cc:2272] T dc5742413fd242b88ca8c4668e6b8144 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.085312 17157 tablet_replica.cc:333] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.085879 17157 raft_consensus.cc:2243] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.086516 17157 raft_consensus.cc:2272] T 57dbd4858e8f41a2af67d6de6eae9928 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.088907 17157 tablet_replica.cc:333] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.089519 17157 raft_consensus.cc:2243] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.090161 17157 raft_consensus.cc:2272] T d8bf6d66a261490caa69a80b24ea015f P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.092509 17157 tablet_replica.cc:333] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.093077 17157 raft_consensus.cc:2243] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.093703 17157 raft_consensus.cc:2272] T 6021c985b4e44fe880350a321132a9b5 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.096041 17157 tablet_replica.cc:333] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.096583 17157 raft_consensus.cc:2243] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.097767 17157 raft_consensus.cc:2272] T 0936d54d511a4cc3903264974a56ecb7 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.100172 17157 tablet_replica.cc:333] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.100854 17157 raft_consensus.cc:2243] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.101747 17157 raft_consensus.cc:2272] T b9f2e3c205c14e43b532c52bc7fa4c90 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.103915 17157 tablet_replica.cc:333] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.104542 17157 raft_consensus.cc:2243] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.105242 17157 raft_consensus.cc:2272] T fafcfc4614304a81b0eadc8f445bfdb8 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.107391 17157 tablet_replica.cc:333] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.107919 17157 raft_consensus.cc:2243] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.108460 17157 raft_consensus.cc:2272] T 2e4622d18b474b659ebea19e9aad0d05 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.110728 17157 tablet_replica.cc:333] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.111311 17157 raft_consensus.cc:2243] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.112412 17157 raft_consensus.cc:2272] T a971a418238d45d288b49960c0abeea0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.115219 17157 tablet_replica.cc:333] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.116114 17157 raft_consensus.cc:2243] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.116819 17157 raft_consensus.cc:2272] T 37ef0235d410491ab47198988075f6e4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.119820 17157 tablet_replica.cc:333] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.120429 17157 raft_consensus.cc:2243] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.121320 17157 raft_consensus.cc:2272] T c5398d6474dd4eed946f45af9214825c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.123481 17157 tablet_replica.cc:333] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.124039 17157 raft_consensus.cc:2243] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Raft consensus shutting down.
I20260709 12:20:15.124436 17157 pending_rounds.cc:70] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Trying to abort 1 pending ops.
I20260709 12:20:15.124662 17157 pending_rounds.cc:77] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7305623238937452544 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "ca383fa1243547c3b32dfbd6857a3fba" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } } new_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e23c0ce8d89940da8eac6a6002e349c5" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 37771 } } peers { permanent_uuid: "3da7ad8fa20a4591963b660e4c848e7f" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 44577 } attrs { replace: true } } peers { permanent_uuid: "177254018ba7448aa2b9110c9c9be9e3" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 40227 } } peers { permanent_uuid: "991dcd2a70c74fdb89ca9494cc09fb7b" member_type: NON_VOTER last_known_addr { host: "127.16.193.68" port: 43821 } attrs { promote: true } } } }
I20260709 12:20:15.125595 17157 raft_consensus.cc:2917] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 LEARNER]: Aborting config change with OpId 1.2: Aborted: Op aborted
I20260709 12:20:15.125957 17157 raft_consensus.cc:2889] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:15.126214 17157 raft_consensus.cc:2272] T ca383fa1243547c3b32dfbd6857a3fba P 991dcd2a70c74fdb89ca9494cc09fb7b [term 2 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:20:15.128418 17157 tablet_replica.cc:333] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.128907 17157 raft_consensus.cc:2243] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.129407 17157 raft_consensus.cc:2272] T 7dd86332c91f4c8d8616b22b5daaf743 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.131455 17157 tablet_replica.cc:333] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.131986 17157 raft_consensus.cc:2243] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.132493 17157 raft_consensus.cc:2272] T e7d0369b4aea4afb960bf859e9b7738c P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.134631 17157 tablet_replica.cc:333] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.135154 17157 raft_consensus.cc:2243] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.136053 17157 raft_consensus.cc:2272] T 9e2ee63a449648bfb064fadee177cadb P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.138257 17157 tablet_replica.cc:333] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.138856 17157 raft_consensus.cc:2243] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.139879 17157 raft_consensus.cc:2272] T 548b719daf794fb881c5606ebec2abb3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.142140 17157 tablet_replica.cc:333] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.142799 17157 raft_consensus.cc:2243] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.143689 17157 raft_consensus.cc:2272] T 3153d259c2254b2d9bdc71d8322cafc0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.146586 17157 tablet_replica.cc:333] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.147214 17157 raft_consensus.cc:2243] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.147850 17157 raft_consensus.cc:2272] T adea31c273544d04b7f16f9b9a3ae610 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.150142 17157 tablet_replica.cc:333] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.150646 17157 raft_consensus.cc:2243] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.151576 17157 raft_consensus.cc:2272] T 389aa517d744435bb043ff8c52d86c2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.153622 17157 tablet_replica.cc:333] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.154101 17157 raft_consensus.cc:2243] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.154789 17157 raft_consensus.cc:2272] T 6ab0a3f9b34149918a57aec5d20b4884 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.156661 17157 tablet_replica.cc:333] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.157258 17157 raft_consensus.cc:2243] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.157784 17157 raft_consensus.cc:2272] T 5461f19f144642beb790dcf7f1515a2d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.159754 17157 tablet_replica.cc:333] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.160236 17157 raft_consensus.cc:2243] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.160725 17157 raft_consensus.cc:2272] T dc49a9d4c95c46f38ca4a279c913204d P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.162955 17157 tablet_replica.cc:333] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.163565 17157 raft_consensus.cc:2243] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.164937 17157 raft_consensus.cc:2272] T 5d4d1763576e4c49849611360f37665a P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:15.167251 19009 consensus_peers.cc:597] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 3da7ad8fa20a4591963b660e4c848e7f (127.16.193.66:44577): Couldn't send request to peer 3da7ad8fa20a4591963b660e4c848e7f. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:44577: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:15.167555 17157 tablet_replica.cc:333] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.168179 17157 raft_consensus.cc:2243] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.169202 17157 raft_consensus.cc:2272] T c749b52c1c8c44b78cee98b4b4d2ef29 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.171501 17157 tablet_replica.cc:333] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.172092 17157 raft_consensus.cc:2243] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.172704 17157 raft_consensus.cc:2272] T ea246fa8cc7e4edbb136da2ee92fb1d3 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.175446 17157 tablet_replica.cc:333] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.175940 17157 raft_consensus.cc:2243] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.176478 17157 raft_consensus.cc:2272] T 0931a54c049145dfad672173c0b29a40 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.178381 17157 tablet_replica.cc:333] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.178901 17157 raft_consensus.cc:2243] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.179832 17157 raft_consensus.cc:2272] T ad852e5220a14f4898d2f069a5fb4a63 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.182193 17157 tablet_replica.cc:333] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.182678 17157 raft_consensus.cc:2243] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.183617 17157 raft_consensus.cc:2272] T 8bc6be8433ea4f43a3c0ac6dfe27398e P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.185781 17157 tablet_replica.cc:333] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.186522 17157 raft_consensus.cc:2243] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.187613 17157 raft_consensus.cc:2272] T cc5f8db079f74a52b906175ad2c4cb56 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.190657 17157 tablet_replica.cc:333] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
W20260709 12:20:15.190654 19007 consensus_peers.cc:597] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b -> Peer 177254018ba7448aa2b9110c9c9be9e3 (127.16.193.67:40227): Couldn't send request to peer 177254018ba7448aa2b9110c9c9be9e3. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:40227: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:15.191411 17157 raft_consensus.cc:2243] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.192019 17157 raft_consensus.cc:2272] T 8e62e9fad92b449d9315da4aa77082b4 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.194113 17157 tablet_replica.cc:333] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.194670 17157 raft_consensus.cc:2243] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.195683 17157 raft_consensus.cc:2272] T 353cbd244a354830a4e903a3b1be2b89 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.197810 17157 tablet_replica.cc:333] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.198356 17157 raft_consensus.cc:2243] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.199275 17157 raft_consensus.cc:2272] T 0071c33745ae42e48eca14d86aa5fac6 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.201349 17157 tablet_replica.cc:333] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.201879 17157 raft_consensus.cc:2243] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.202849 17157 raft_consensus.cc:2272] T 80bc05a330c64471aaa556af75bea031 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.204783 17157 tablet_replica.cc:333] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.205566 17157 raft_consensus.cc:2243] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.206621 17157 raft_consensus.cc:2272] T 35973fcee55042b284c6f62e2681c304 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.208662 17157 tablet_replica.cc:333] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.209213 17157 raft_consensus.cc:2243] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.209769 17157 raft_consensus.cc:2272] T e98a95c8e3c14bcfa51b054f4829293b P 991dcd2a70c74fdb89ca9494cc09fb7b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.211781 17157 tablet_replica.cc:333] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.212266 17157 raft_consensus.cc:2243] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:15.213171 17157 raft_consensus.cc:2272] T 4b135a527cce4b14b789d23618d411b0 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.215350 17157 tablet_replica.cc:333] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.215792 17157 raft_consensus.cc:2243] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.216307 17157 raft_consensus.cc:2272] T 80f93a4f5d0e45dd8b049fe76d3a0fda P 991dcd2a70c74fdb89ca9494cc09fb7b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.218468 17157 tablet_replica.cc:333] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b: stopping tablet replica
I20260709 12:20:15.218883 17157 raft_consensus.cc:2243] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:15.219367 17157 raft_consensus.cc:2272] T d266f68528964b1981ffe1df49861b08 P 991dcd2a70c74fdb89ca9494cc09fb7b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:15.761633 17157 tablet_server.cc:196] TabletServer@127.16.193.68:0 shutdown complete.
I20260709 12:20:15.845067 17157 master.cc:562] Master@127.16.193.126:46027 shutting down...
W20260709 12:20:15.882970 18246 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:20:15.883368 18246 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:20:15.883587 18246 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:20:25.057364 18182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111) [suppressed 327 similar messages]
W20260709 12:20:25.061129 18182 catalog_manager.cc:4755] TS e23c0ce8d89940da8eac6a6002e349c5 (127.16.193.65:37771): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 1d0efe89255942c4bd576ff4b96cd91d: Network error: Client connection negotiation failed: client connection to 127.16.193.65:37771: connect: Connection refused (error 111)
I20260709 12:20:26.785063 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:26.786322 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:26.786752 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 79114be45f9144e78102be8765ae6af5: stopping tablet replica
I20260709 12:20:27.007845 17157 master.cc:584] Master@127.16.193.126:46027 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (254996 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:20:27.126844 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:33125
I20260709 12:20:27.127990 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:27.134239 26273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.134806 26276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.134321 26274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.136026 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:27.140144 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:27.140340 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.140556 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599627140519 us; error 0 us; skew 500 ppm
I20260709 12:20:27.141139 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.143920 17157 webserver.cc:533] Webserver started at http://127.16.193.126:33525/ using document root <none> and password file <none>
I20260709 12:20:27.144446 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.144673 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.144990 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.146319 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "718f20e901874dc78478044b5f19e76a"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.151000 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:20:27.154500 26281 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.155325 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:27.155658 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "718f20e901874dc78478044b5f19e76a"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.155992 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.180683 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.182047 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:27.224575 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:33125
I20260709 12:20:27.224704 26332 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:33125 every 8 connection(s)
I20260709 12:20:27.228924 26333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:27.239997 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a: Bootstrap starting.
I20260709 12:20:27.244990 26333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:27.249882 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a: No bootstrap required, opened a new log
I20260709 12:20:27.252221 26333 raft_consensus.cc:359] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER }
I20260709 12:20:27.252734 26333 raft_consensus.cc:385] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:27.252957 26333 raft_consensus.cc:740] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718f20e901874dc78478044b5f19e76a, State: Initialized, Role: FOLLOWER
I20260709 12:20:27.253521 26333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER }
I20260709 12:20:27.253965 26333 raft_consensus.cc:399] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:27.254186 26333 raft_consensus.cc:493] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:27.254482 26333 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:27.259047 26333 raft_consensus.cc:515] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER }
I20260709 12:20:27.259599 26333 leader_election.cc:304] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 718f20e901874dc78478044b5f19e76a; no voters: 
I20260709 12:20:27.260695 26333 leader_election.cc:290] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:27.261042 26336 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:27.262449 26336 raft_consensus.cc:697] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 1 LEADER]: Becoming Leader. State: Replica: 718f20e901874dc78478044b5f19e76a, State: Running, Role: LEADER
I20260709 12:20:27.263139 26336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER }
I20260709 12:20:27.263710 26333 sys_catalog.cc:565] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:27.270721 26338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718f20e901874dc78478044b5f19e76a. Latest consensus state: current_term: 1 leader_uuid: "718f20e901874dc78478044b5f19e76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER } }
I20260709 12:20:27.270946 26337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "718f20e901874dc78478044b5f19e76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718f20e901874dc78478044b5f19e76a" member_type: VOTER } }
I20260709 12:20:27.271565 26338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:27.271751 26337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:27.273425 26346 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:27.278266 26346 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:27.279774 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:27.287338 26346 catalog_manager.cc:1383] Generated new cluster ID: 81f851fe7ace4920af01ab94a0bb4869
I20260709 12:20:27.287596 26346 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:27.311117 26346 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:27.312692 26346 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:27.324328 26346 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a: Generated new TSK 0
I20260709 12:20:27.324898 26346 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:27.346271 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:27.352432 26354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.353807 26355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.355540 26357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.356077 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:27.358755 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:27.358935 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.359054 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599627359043 us; error 0 us; skew 500 ppm
I20260709 12:20:27.359601 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.361817 17157 webserver.cc:533] Webserver started at http://127.16.193.65:37971/ using document root <none> and password file <none>
I20260709 12:20:27.362236 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.362387 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.362591 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.363829 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "fc524c91d3c249c0aeb08dae83b98e41"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.368331 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:20:27.371745 26362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.372560 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:20:27.372864 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "fc524c91d3c249c0aeb08dae83b98e41"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.373150 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.437486 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.438710 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:27.440254 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:27.442516 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:27.442713 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.442962 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:27.443130 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.485478 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:41581
I20260709 12:20:27.485606 26424 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:41581 every 8 connection(s)
I20260709 12:20:27.491401 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:27.502517 26429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.504136 26430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.506726 26432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.507741 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:27.510474 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:27.510699 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.510901 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599627510882 us; error 0 us; skew 500 ppm
I20260709 12:20:27.511668 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.513069 26425 heartbeater.cc:344] Connected to a master server at 127.16.193.126:33125
I20260709 12:20:27.513537 26425 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:27.514585 17157 webserver.cc:533] Webserver started at http://127.16.193.66:39893/ using document root <none> and password file <none>
I20260709 12:20:27.514673 26425 heartbeater.cc:507] Master 127.16.193.126:33125 requested a full tablet report, sending...
I20260709 12:20:27.515374 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.515655 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.515992 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.517356 26298 ts_manager.cc:194] Registered new tserver with Master: fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:27.517438 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "bb83296e641848deaddf424cc59fda0e"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.519201 26298 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44418
I20260709 12:20:27.522781 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:20:27.526573 26437 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.527434 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:20:27.527732 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "bb83296e641848deaddf424cc59fda0e"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.528007 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.549175 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.550443 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:27.551992 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:27.554239 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:27.554421 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.554700 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:27.554870 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.595925 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:37205
I20260709 12:20:27.596030 26499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:37205 every 8 connection(s)
I20260709 12:20:27.601068 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:27.610948 26504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.612164 26503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.614828 26506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.615401 26500 heartbeater.cc:344] Connected to a master server at 127.16.193.126:33125
I20260709 12:20:27.615679 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:27.615921 26500 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:27.616689 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:27.616928 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.616838 26500 heartbeater.cc:507] Master 127.16.193.126:33125 requested a full tablet report, sending...
I20260709 12:20:27.617309 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599627617271 us; error 0 us; skew 500 ppm
I20260709 12:20:27.618018 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.619426 26298 ts_manager.cc:194] Registered new tserver with Master: bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:27.620870 17157 webserver.cc:533] Webserver started at http://127.16.193.67:33283/ using document root <none> and password file <none>
I20260709 12:20:27.621389 26298 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44426
I20260709 12:20:27.621587 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.621840 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.622146 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.623557 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "297900273c7740af9938749b3ce3a80d"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.627972 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:27.631587 26511 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.632432 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:20:27.632740 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "297900273c7740af9938749b3ce3a80d"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.633064 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.661223 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.662561 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:27.663898 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:27.666148 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:27.666374 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.666607 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:27.666778 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.708041 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:41389
I20260709 12:20:27.708161 26573 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:41389 every 8 connection(s)
I20260709 12:20:27.712970 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:27.720862 26577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.722440 26578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:27.724560 26580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:27.725375 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:27.726814 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:20:27.726853 26574 heartbeater.cc:344] Connected to a master server at 127.16.193.126:33125
W20260709 12:20:27.727095 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:27.727403 26574 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:27.727460 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599627727430 us; error 0 us; skew 500 ppm
I20260709 12:20:27.728323 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:27.728374 26574 heartbeater.cc:507] Master 127.16.193.126:33125 requested a full tablet report, sending...
I20260709 12:20:27.730778 26298 ts_manager.cc:194] Registered new tserver with Master: 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:27.731138 17157 webserver.cc:533] Webserver started at http://127.16.193.68:37095/ using document root <none> and password file <none>
I20260709 12:20:27.731739 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:27.731927 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:27.732203 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:27.732493 26298 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44442
I20260709 12:20:27.733379 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-3-root/instance:
uuid: "39930cd9790d4943bdec6f9df4ce2641"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.739080 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:20:27.742542 26585 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.743475 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:20:27.743783 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
uuid: "39930cd9790d4943bdec6f9df4ce2641"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:27.744040 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:27.761668 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:27.763099 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:27.764456 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:27.766690 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:27.766925 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:27.767161 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:27.767408 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:20:27.809638 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.68:44181
I20260709 12:20:27.809743 26647 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.68:44181 every 8 connection(s)
I20260709 12:20:27.824205 26648 heartbeater.cc:344] Connected to a master server at 127.16.193.126:33125
I20260709 12:20:27.824695 26648 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:27.825584 26648 heartbeater.cc:507] Master 127.16.193.126:33125 requested a full tablet report, sending...
I20260709 12:20:27.827629 26298 ts_manager.cc:194] Registered new tserver with Master: 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:27.828125 17157 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015165218s
I20260709 12:20:27.829128 26298 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44452
I20260709 12:20:27.854105 26298 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44458:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:20:28.090981 26390 tablet_service.cc:1511] Processing CreateTablet for tablet 8f22e34b989c46e1867151870954a0e5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:28.092578 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f22e34b989c46e1867151870954a0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.097812 26389 tablet_service.cc:1511] Processing CreateTablet for tablet 4358c31532834ddd8a1e690e2accadfb (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:28.099846 26389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4358c31532834ddd8a1e690e2accadfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.107153 26465 tablet_service.cc:1511] Processing CreateTablet for tablet 8f22e34b989c46e1867151870954a0e5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:28.108675 26465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f22e34b989c46e1867151870954a0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.113042 26388 tablet_service.cc:1511] Processing CreateTablet for tablet 237bd2b722334bda9aeb29035546ca4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:28.114511 26388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 237bd2b722334bda9aeb29035546ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.117450 26387 tablet_service.cc:1511] Processing CreateTablet for tablet 32add6893dda4c028a237f05c1f1ec81 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:28.118758 26387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32add6893dda4c028a237f05c1f1ec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.122732 26386 tablet_service.cc:1511] Processing CreateTablet for tablet df382f80de9947869b3760fc62e30a77 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:28.124171 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df382f80de9947869b3760fc62e30a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.127902 26385 tablet_service.cc:1511] Processing CreateTablet for tablet f97b53680d1349fa8b2bb3c5e067602d (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:28.129304 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97b53680d1349fa8b2bb3c5e067602d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.130018 26464 tablet_service.cc:1511] Processing CreateTablet for tablet 4358c31532834ddd8a1e690e2accadfb (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:28.132055 26464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4358c31532834ddd8a1e690e2accadfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.133973 26463 tablet_service.cc:1511] Processing CreateTablet for tablet c772c40b55fc4ee88cce69a147fd7ea9 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:28.135537 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c772c40b55fc4ee88cce69a147fd7ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.136636 26538 tablet_service.cc:1511] Processing CreateTablet for tablet 237bd2b722334bda9aeb29035546ca4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:28.138339 26538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 237bd2b722334bda9aeb29035546ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.135098 26539 tablet_service.cc:1511] Processing CreateTablet for tablet 8f22e34b989c46e1867151870954a0e5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:28.142069 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f22e34b989c46e1867151870954a0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.143740 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 4358c31532834ddd8a1e690e2accadfb (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:28.145660 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4358c31532834ddd8a1e690e2accadfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.154189 26389 tablet_service.cc:1511] Processing CreateTablet for tablet 759e0f60b4604c2e9abda3833a0e9e6f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:28.156082 26389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759e0f60b4604c2e9abda3833a0e9e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.157456 26612 tablet_service.cc:1511] Processing CreateTablet for tablet 237bd2b722334bda9aeb29035546ca4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:28.159430 26612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 237bd2b722334bda9aeb29035546ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.168787 26611 tablet_service.cc:1511] Processing CreateTablet for tablet c772c40b55fc4ee88cce69a147fd7ea9 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:28.170282 26667 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:28.170475 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c772c40b55fc4ee88cce69a147fd7ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.176688 26462 tablet_service.cc:1511] Processing CreateTablet for tablet 32add6893dda4c028a237f05c1f1ec81 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:28.178162 26462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32add6893dda4c028a237f05c1f1ec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.188987 26537 tablet_service.cc:1511] Processing CreateTablet for tablet c772c40b55fc4ee88cce69a147fd7ea9 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:28.190622 26537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c772c40b55fc4ee88cce69a147fd7ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.193220 26387 tablet_service.cc:1511] Processing CreateTablet for tablet 4125b700ff114ebeb4ff2849edf5e291 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:28.194532 26387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125b700ff114ebeb4ff2849edf5e291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.200948 26610 tablet_service.cc:1511] Processing CreateTablet for tablet df382f80de9947869b3760fc62e30a77 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:28.202452 26610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df382f80de9947869b3760fc62e30a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.202618 26535 tablet_service.cc:1511] Processing CreateTablet for tablet f97b53680d1349fa8b2bb3c5e067602d (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:28.200948 26536 tablet_service.cc:1511] Processing CreateTablet for tablet 32add6893dda4c028a237f05c1f1ec81 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:28.204023 26535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97b53680d1349fa8b2bb3c5e067602d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.207638 26536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32add6893dda4c028a237f05c1f1ec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.209998 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 85165efa340d47638ef75f49dda75e8a (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:28.213238 26534 tablet_service.cc:1511] Processing CreateTablet for tablet a9607fece157456e9d07bf4ef22d370c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:28.214663 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9607fece157456e9d07bf4ef22d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.219337 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85165efa340d47638ef75f49dda75e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.220455 26533 tablet_service.cc:1511] Processing CreateTablet for tablet 759e0f60b4604c2e9abda3833a0e9e6f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:28.221400 26461 tablet_service.cc:1511] Processing CreateTablet for tablet df382f80de9947869b3760fc62e30a77 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:28.221861 26533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759e0f60b4604c2e9abda3833a0e9e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.222797 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df382f80de9947869b3760fc62e30a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.227977 26532 tablet_service.cc:1511] Processing CreateTablet for tablet 85165efa340d47638ef75f49dda75e8a (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:28.229288 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85165efa340d47638ef75f49dda75e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.230378 26667 tablet_bootstrap.cc:654] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.232990 26460 tablet_service.cc:1511] Processing CreateTablet for tablet a9607fece157456e9d07bf4ef22d370c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:28.234436 26460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9607fece157456e9d07bf4ef22d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.245865 26609 tablet_service.cc:1511] Processing CreateTablet for tablet f97b53680d1349fa8b2bb3c5e067602d (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:28.247095 26459 tablet_service.cc:1511] Processing CreateTablet for tablet 759e0f60b4604c2e9abda3833a0e9e6f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:28.248553 26459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759e0f60b4604c2e9abda3833a0e9e6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.249385 26608 tablet_service.cc:1511] Processing CreateTablet for tablet a9607fece157456e9d07bf4ef22d370c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:28.252388 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 85165efa340d47638ef75f49dda75e8a (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:28.252853 26458 tablet_service.cc:1511] Processing CreateTablet for tablet 4125b700ff114ebeb4ff2849edf5e291 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:28.254199 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125b700ff114ebeb4ff2849edf5e291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.254989 26607 tablet_service.cc:1511] Processing CreateTablet for tablet 4125b700ff114ebeb4ff2849edf5e291 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:28.256500 26609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97b53680d1349fa8b2bb3c5e067602d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.259786 26608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9607fece157456e9d07bf4ef22d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.263671 26607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125b700ff114ebeb4ff2849edf5e291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.267649 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85165efa340d47638ef75f49dda75e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.267427 26531 tablet_service.cc:1511] Processing CreateTablet for tablet 0822222bd071434cb1c7ec59559e02a6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:28.267355 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 0822222bd071434cb1c7ec59559e02a6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:28.269004 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0822222bd071434cb1c7ec59559e02a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.269004 26531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0822222bd071434cb1c7ec59559e02a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:28.275084 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:28.293870 26605 tablet_service.cc:1511] Processing CreateTablet for tablet 0822222bd071434cb1c7ec59559e02a6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:28.295535 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0822222bd071434cb1c7ec59559e02a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.331655 26668 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:28.341338 26385 tablet_service.cc:1511] Processing CreateTablet for tablet 429d15dc784e4494ad111c92f4812cc6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:28.342885 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429d15dc784e4494ad111c92f4812cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.355814 26460 tablet_service.cc:1511] Processing CreateTablet for tablet 429d15dc784e4494ad111c92f4812cc6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:28.357319 26460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429d15dc784e4494ad111c92f4812cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.360152 26532 tablet_service.cc:1511] Processing CreateTablet for tablet 429d15dc784e4494ad111c92f4812cc6 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:28.361625 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429d15dc784e4494ad111c92f4812cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.379285 26669 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:28.380028 26670 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:28.385381 26606 tablet_service.cc:1511] Processing CreateTablet for tablet fe3aff16fd4242f5a9ef1a3038bbe22f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:28.386770 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3aff16fd4242f5a9ef1a3038bbe22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.385895 26669 tablet_bootstrap.cc:654] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.400208 26461 tablet_service.cc:1511] Processing CreateTablet for tablet fe3aff16fd4242f5a9ef1a3038bbe22f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:28.403676 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3aff16fd4242f5a9ef1a3038bbe22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.395860 26386 tablet_service.cc:1511] Processing CreateTablet for tablet fe3aff16fd4242f5a9ef1a3038bbe22f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:28.410394 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe3aff16fd4242f5a9ef1a3038bbe22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.421120 26668 tablet_bootstrap.cc:654] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.426167 26667 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:28.426707 26667 ts_tablet_manager.cc:1403] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.257s	user 0.007s	sys 0.038s
I20260709 12:20:28.430094 26667 raft_consensus.cc:359] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.430910 26667 raft_consensus.cc:385] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.431285 26667 raft_consensus.cc:740] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.431078 26534 tablet_service.cc:1511] Processing CreateTablet for tablet b00aa72f3c874a77b2d226ecbebbe834 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:28.432821 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00aa72f3c874a77b2d226ecbebbe834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.432448 26667 consensus_queue.cc:260] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.446594 26670 tablet_bootstrap.cc:654] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.446691 26607 tablet_service.cc:1511] Processing CreateTablet for tablet b00aa72f3c874a77b2d226ecbebbe834 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:28.448588 26669 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:28.451743 26669 ts_tablet_manager.cc:1403] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.034s
I20260709 12:20:28.452020 26607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00aa72f3c874a77b2d226ecbebbe834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.454986 26669 raft_consensus.cc:359] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.457736 26669 raft_consensus.cc:385] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.458511 26669 raft_consensus.cc:740] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.449208 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc5473d41af4ddc9dd47559842556ca (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:28.449486 26385 tablet_service.cc:1511] Processing CreateTablet for tablet b00aa72f3c874a77b2d226ecbebbe834 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:28.462385 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc5473d41af4ddc9dd47559842556ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.460057 26539 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc5473d41af4ddc9dd47559842556ca (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:28.462616 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00aa72f3c874a77b2d226ecbebbe834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.464512 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc5473d41af4ddc9dd47559842556ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.465003 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc5473d41af4ddc9dd47559842556ca (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:28.466279 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc5473d41af4ddc9dd47559842556ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.460058 26669 consensus_queue.cc:260] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.463203 26668 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:28.477942 26668 ts_tablet_manager.cc:1403] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.155s	user 0.016s	sys 0.000s
I20260709 12:20:28.478775 26574 heartbeater.cc:499] Master 127.16.193.126:33125 was elected leader, sending a full tablet report...
I20260709 12:20:28.481016 26669 ts_tablet_manager.cc:1434] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.003s
I20260709 12:20:28.482627 26386 tablet_service.cc:1511] Processing CreateTablet for tablet bce90f860c124766af2e9d46c5864016 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:28.484019 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce90f860c124766af2e9d46c5864016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.488691 26669 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:28.496297 26458 tablet_service.cc:1511] Processing CreateTablet for tablet bce90f860c124766af2e9d46c5864016 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:28.497763 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce90f860c124766af2e9d46c5864016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.504813 26668 raft_consensus.cc:359] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.505651 26668 raft_consensus.cc:385] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.506016 26668 raft_consensus.cc:740] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.516317 26425 heartbeater.cc:499] Master 127.16.193.126:33125 was elected leader, sending a full tablet report...
I20260709 12:20:28.517223 26667 ts_tablet_manager.cc:1434] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.020s
I20260709 12:20:28.518741 26669 tablet_bootstrap.cc:654] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.506934 26668 consensus_queue.cc:260] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.521560 26667 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:28.526751 26605 tablet_service.cc:1511] Processing CreateTablet for tablet ba40df2a50424f46b52f9c4cf6894664 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:28.528180 26539 tablet_service.cc:1511] Processing CreateTablet for tablet bce90f860c124766af2e9d46c5864016 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:28.528944 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40df2a50424f46b52f9c4cf6894664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.529507 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce90f860c124766af2e9d46c5864016. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.537329 26667 tablet_bootstrap.cc:654] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.542299 26457 tablet_service.cc:1511] Processing CreateTablet for tablet ba40df2a50424f46b52f9c4cf6894664 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:28.543771 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40df2a50424f46b52f9c4cf6894664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.544714 26500 heartbeater.cc:499] Master 127.16.193.126:33125 was elected leader, sending a full tablet report...
I20260709 12:20:28.547101 26670 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:28.547803 26670 ts_tablet_manager.cc:1403] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.168s	user 0.034s	sys 0.037s
I20260709 12:20:28.551082 26670 raft_consensus.cc:359] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.544420 26386 tablet_service.cc:1511] Processing CreateTablet for tablet ba40df2a50424f46b52f9c4cf6894664 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:28.557634 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40df2a50424f46b52f9c4cf6894664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.556461 26670 raft_consensus.cc:385] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.558578 26670 raft_consensus.cc:740] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.560892 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 97abcb3c90494ee2b8ad8f45c34923f5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:28.559701 26670 consensus_queue.cc:260] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.564409 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97abcb3c90494ee2b8ad8f45c34923f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.561518 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 97abcb3c90494ee2b8ad8f45c34923f5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:28.568290 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97abcb3c90494ee2b8ad8f45c34923f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.572219 26668 ts_tablet_manager.cc:1434] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.094s	user 0.011s	sys 0.000s
I20260709 12:20:28.573254 26668 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:28.589059 26385 tablet_service.cc:1511] Processing CreateTablet for tablet 97abcb3c90494ee2b8ad8f45c34923f5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:28.593021 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97abcb3c90494ee2b8ad8f45c34923f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.594491 26458 tablet_service.cc:1511] Processing CreateTablet for tablet 157df6bf4a63423eb6f1c4f7de58fb9b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:28.596376 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157df6bf4a63423eb6f1c4f7de58fb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.600606 26667 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:28.601228 26667 ts_tablet_manager.cc:1403] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.016s
W20260709 12:20:28.602226 26501 tablet.cc:2406] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:28.612303 26670 ts_tablet_manager.cc:1434] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.064s	user 0.013s	sys 0.008s
I20260709 12:20:28.614229 26669 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:28.616954 26669 ts_tablet_manager.cc:1403] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.129s	user 0.022s	sys 0.022s
I20260709 12:20:28.619459 26648 heartbeater.cc:499] Master 127.16.193.126:33125 was elected leader, sending a full tablet report...
I20260709 12:20:28.621848 26669 raft_consensus.cc:359] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:28.622611 26669 raft_consensus.cc:385] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.622895 26669 raft_consensus.cc:740] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.623659 26669 consensus_queue.cc:260] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:28.625730 26669 ts_tablet_manager.cc:1434] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:20:28.626551 26669 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:28.629622 26676 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.630177 26676 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:28.634043 26668 tablet_bootstrap.cc:654] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.648968 26670 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:28.653098 26673 raft_consensus.cc:493] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.653616 26673 raft_consensus.cc:515] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.653872 26667 raft_consensus.cc:359] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.654697 26667 raft_consensus.cc:385] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.655023 26667 raft_consensus.cc:740] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.655838 26667 consensus_queue.cc:260] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.656606 26605 tablet_service.cc:1511] Processing CreateTablet for tablet 899048d400bc4540a891c445bcd9bd25 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:28.657047 26460 tablet_service.cc:1511] Processing CreateTablet for tablet 84283d0aa91944d482d29b9d70c80299 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:28.657955 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899048d400bc4540a891c445bcd9bd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.658315 26460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84283d0aa91944d482d29b9d70c80299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.645359 26676 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:28.666766 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 84283d0aa91944d482d29b9d70c80299 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:28.667057 26387 tablet_service.cc:1511] Processing CreateTablet for tablet 899048d400bc4540a891c445bcd9bd25 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:28.668161 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84283d0aa91944d482d29b9d70c80299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.669373 26387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899048d400bc4540a891c445bcd9bd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.629671 26539 tablet_service.cc:1511] Processing CreateTablet for tablet 157df6bf4a63423eb6f1c4f7de58fb9b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:28.660539 26676 raft_consensus.cc:493] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.685818 26667 ts_tablet_manager.cc:1434] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.012s
I20260709 12:20:28.673904 26673 leader_election.cc:290] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:28.689316 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 56bca68ee09b4f00a06f68b9fc9ca949 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:28.690841 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bca68ee09b4f00a06f68b9fc9ca949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.690997 26458 tablet_service.cc:1511] Processing CreateTablet for tablet 899048d400bc4540a891c445bcd9bd25 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:28.692322 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899048d400bc4540a891c445bcd9bd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.689379 26607 tablet_service.cc:1511] Processing CreateTablet for tablet 157df6bf4a63423eb6f1c4f7de58fb9b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:28.686476 26676 raft_consensus.cc:515] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.698722 26607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157df6bf4a63423eb6f1c4f7de58fb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.699630 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157df6bf4a63423eb6f1c4f7de58fb9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.699757 26670 tablet_bootstrap.cc:654] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.706737 26463 tablet_service.cc:1511] Processing CreateTablet for tablet bb49253f88584c74979fb16378c007d1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:28.708341 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb49253f88584c74979fb16378c007d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.711620 26669 tablet_bootstrap.cc:654] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.712648 26668 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:28.718492 26388 tablet_service.cc:1511] Processing CreateTablet for tablet 56bca68ee09b4f00a06f68b9fc9ca949 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:28.716667 26668 ts_tablet_manager.cc:1403] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.144s	user 0.009s	sys 0.016s
I20260709 12:20:28.718696 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 84283d0aa91944d482d29b9d70c80299 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:28.719913 26388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bca68ee09b4f00a06f68b9fc9ca949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.720170 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84283d0aa91944d482d29b9d70c80299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.722425 26668 raft_consensus.cc:359] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:28.723167 26668 raft_consensus.cc:385] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.731141 26667 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:28.735186 26676 leader_election.cc:290] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:28.750980 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:28.754001 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c772c40b55fc4ee88cce69a147fd7ea9" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:28.754959 26623 raft_consensus.cc:2468] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
W20260709 12:20:28.756485 26513 leader_election.cc:343] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:28.758065 26513 leader_election.cc:304] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:28.758620 26677 raft_consensus.cc:493] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.759383 26676 raft_consensus.cc:2804] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:28.759167 26677 raft_consensus.cc:515] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.732767 26668 raft_consensus.cc:740] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.771291 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 56bca68ee09b4f00a06f68b9fc9ca949 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:28.772619 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bca68ee09b4f00a06f68b9fc9ca949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.764979 26676 raft_consensus.cc:493] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.779529 26676 raft_consensus.cc:3060] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:28.777456 26668 consensus_queue.cc:260] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:28.778458 26606 tablet_service.cc:1511] Processing CreateTablet for tablet eff188538952450b970105de07be9a85 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:28.764736 26677 leader_election.cc:290] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:28.788446 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff188538952450b970105de07be9a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.796103 26385 tablet_service.cc:1511] Processing CreateTablet for tablet d6eeaddd79764e06b3ac77e3c20ffd49 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:28.796571 26669 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:28.797341 26669 ts_tablet_manager.cc:1403] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.171s	user 0.027s	sys 0.010s
I20260709 12:20:28.797456 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eeaddd79764e06b3ac77e3c20ffd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.800679 26669 raft_consensus.cc:359] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:28.801427 26669 raft_consensus.cc:385] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.801723 26669 raft_consensus.cc:740] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.806205 26613 tablet_service.cc:1511] Processing CreateTablet for tablet bb49253f88584c74979fb16378c007d1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:28.807694 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb49253f88584c74979fb16378c007d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.807749 26670 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:28.809242 26670 ts_tablet_manager.cc:1403] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.002s
I20260709 12:20:28.811733 26457 tablet_service.cc:1511] Processing CreateTablet for tablet d6eeaddd79764e06b3ac77e3c20ffd49 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:28.812994 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eeaddd79764e06b3ac77e3c20ffd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.823469 26386 tablet_service.cc:1511] Processing CreateTablet for tablet eff188538952450b970105de07be9a85 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:28.828447 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff188538952450b970105de07be9a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.836632 26667 tablet_bootstrap.cc:654] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.842629 26670 raft_consensus.cc:359] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.840821 26532 tablet_service.cc:1511] Processing CreateTablet for tablet d6eeaddd79764e06b3ac77e3c20ffd49 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:28.843446 26670 raft_consensus.cc:385] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:28.843817 26670 raft_consensus.cc:740] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:28.841655 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:28.844231 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6eeaddd79764e06b3ac77e3c20ffd49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.844233 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c772c40b55fc4ee88cce69a147fd7ea9" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:28.840471 26533 tablet_service.cc:1511] Processing CreateTablet for tablet bb49253f88584c74979fb16378c007d1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:28.846570 26668 ts_tablet_manager.cc:1434] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.127s	user 0.004s	sys 0.004s
I20260709 12:20:28.847131 26533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb49253f88584c74979fb16378c007d1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:28.849988 26514 leader_election.cc:343] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:28.848567 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:28.859681 26512 leader_election.cc:343] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:28.860340 26611 tablet_service.cc:1511] Processing CreateTablet for tablet bf0a016521b542548a73d6110ed04210 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:28.861744 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0a016521b542548a73d6110ed04210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.844673 26670 consensus_queue.cc:260] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.864910 26670 ts_tablet_manager.cc:1434] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.003s
I20260709 12:20:28.825866 26669 consensus_queue.cc:260] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:28.869166 26463 tablet_service.cc:1511] Processing CreateTablet for tablet eff188538952450b970105de07be9a85 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:28.870566 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff188538952450b970105de07be9a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.877101 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:28.878793 26365 leader_election.cc:343] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:28.879878 26670 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:28.883661 26668 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:28.860328 26512 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:28.902285 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:28.902998 26670 tablet_bootstrap.cc:654] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:28.907147 26387 tablet_service.cc:1511] Processing CreateTablet for tablet bf0a016521b542548a73d6110ed04210 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:28.922075 26676 raft_consensus.cc:515] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.925633 26387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0a016521b542548a73d6110ed04210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.913173 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 7345dd68527a40889384fe8d3710abda (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:28.928958 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7345dd68527a40889384fe8d3710abda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.912685 26388 tablet_service.cc:1511] Processing CreateTablet for tablet f76e69771f3c45fd842cbfca4e7b66d5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:28.930370 26388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f76e69771f3c45fd842cbfca4e7b66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.933045 26460 tablet_service.cc:1511] Processing CreateTablet for tablet f76e69771f3c45fd842cbfca4e7b66d5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:28.934650 26460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f76e69771f3c45fd842cbfca4e7b66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.903043 26400 raft_consensus.cc:2468] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:28.939637 26438 leader_election.cc:304] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:28.940611 26677 raft_consensus.cc:2804] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:28.941015 26677 raft_consensus.cc:493] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.941335 26677 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:28.950223 26677 raft_consensus.cc:515] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:28.952035 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 113f3dbb8bd5466abbfb043ef9cc4d87 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:28.957988 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:28.958650 26399 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:28.959867 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 113f3dbb8bd5466abbfb043ef9cc4d87 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:28.961383 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113f3dbb8bd5466abbfb043ef9cc4d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.963061 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113f3dbb8bd5466abbfb043ef9cc4d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.962962 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 113f3dbb8bd5466abbfb043ef9cc4d87 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:28.970382 26399 raft_consensus.cc:2468] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:28.963896 26677 leader_election.cc:290] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:28.971107 26607 tablet_service.cc:1511] Processing CreateTablet for tablet f94d403c93084635b0e4c70ec3829e1c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:28.907577 26458 tablet_service.cc:1511] Processing CreateTablet for tablet 7345dd68527a40889384fe8d3710abda (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:28.972095 26438 leader_election.cc:304] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:28.973369 26677 raft_consensus.cc:2804] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:28.972532 26607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d403c93084635b0e4c70ec3829e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.974779 26676 leader_election.cc:290] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:28.976063 26676 raft_consensus.cc:2749] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:28.976171 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c772c40b55fc4ee88cce69a147fd7ea9" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:28.976866 26623 raft_consensus.cc:3060] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:28.984508 26623 raft_consensus.cc:2468] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:28.985774 26513 leader_election.cc:304] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:28.986724 26676 raft_consensus.cc:2804] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:28.987290 26676 raft_consensus.cc:697] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:28.987080 26389 tablet_service.cc:1511] Processing CreateTablet for tablet f94d403c93084635b0e4c70ec3829e1c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:28.988628 26389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d403c93084635b0e4c70ec3829e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:28.988423 26676 consensus_queue.cc:237] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:28.990609 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c772c40b55fc4ee88cce69a147fd7ea9" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
W20260709 12:20:28.992156 26514 leader_election.cc:343] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:28.993536 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7345dd68527a40889384fe8d3710abda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.014945 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:29.012002 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 113f3dbb8bd5466abbfb043ef9cc4d87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.019760 26439 leader_election.cc:343] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.020864 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:29.021404 26670 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
W20260709 12:20:29.010936 26512 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 151431us
W20260709 12:20:29.027978 26366 leader_election.cc:343] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.028425 26366 leader_election.cc:304] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:29.022015 26670 ts_tablet_manager.cc:1403] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.142s	user 0.013s	sys 0.007s
I20260709 12:20:29.029547 26673 raft_consensus.cc:2749] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:29.016541 26677 raft_consensus.cc:697] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
W20260709 12:20:29.022230 26439 leader_election.cc:343] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.022861 26463 tablet_service.cc:1511] Processing CreateTablet for tablet 269623ffea96490eab12418e6cbd2c01 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:29.033717 26533 tablet_service.cc:1511] Processing CreateTablet for tablet bf0a016521b542548a73d6110ed04210 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:29.034191 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 269623ffea96490eab12418e6cbd2c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.034999 26533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0a016521b542548a73d6110ed04210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.037881 26677 consensus_queue.cc:237] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.038280 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 269623ffea96490eab12418e6cbd2c01 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:29.040586 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 269623ffea96490eab12418e6cbd2c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.040940 26461 tablet_service.cc:1511] Processing CreateTablet for tablet c141b67bfa314acca1c0c3c29d3eaac8 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:29.042171 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141b67bfa314acca1c0c3c29d3eaac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.038774 26390 tablet_service.cc:1511] Processing CreateTablet for tablet c141b67bfa314acca1c0c3c29d3eaac8 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:29.045011 26670 raft_consensus.cc:359] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.048067 26668 tablet_bootstrap.cc:654] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.052134 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141b67bfa314acca1c0c3c29d3eaac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.057557 26605 tablet_service.cc:1511] Processing CreateTablet for tablet 81dffbd4eb274c0595db3619e8eb2058 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:29.058868 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dffbd4eb274c0595db3619e8eb2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.059824 26669 ts_tablet_manager.cc:1434] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.262s	user 0.016s	sys 0.008s
I20260709 12:20:29.060982 26669 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:29.065402 26535 tablet_service.cc:1511] Processing CreateTablet for tablet 269623ffea96490eab12418e6cbd2c01 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:29.065060 26539 tablet_service.cc:1511] Processing CreateTablet for tablet f94d403c93084635b0e4c70ec3829e1c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:29.065968 26297 catalog_manager.cc:5697] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:29.066861 26535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 269623ffea96490eab12418e6cbd2c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.068105 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d403c93084635b0e4c70ec3829e1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.071151 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 81dffbd4eb274c0595db3619e8eb2058 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:29.072780 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dffbd4eb274c0595db3619e8eb2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.048548 26670 raft_consensus.cc:385] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.076442 26532 tablet_service.cc:1511] Processing CreateTablet for tablet 7345dd68527a40889384fe8d3710abda (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:29.074986 26531 tablet_service.cc:1511] Processing CreateTablet for tablet c141b67bfa314acca1c0c3c29d3eaac8 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:29.084453 26667 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:29.085173 26465 tablet_service.cc:1511] Processing CreateTablet for tablet 81dffbd4eb274c0595db3619e8eb2058 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:29.093763 26676 raft_consensus.cc:493] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.075157 26670 raft_consensus.cc:740] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.097569 26667 ts_tablet_manager.cc:1403] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.381s	user 0.015s	sys 0.020s
I20260709 12:20:29.096138 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7345dd68527a40889384fe8d3710abda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.097155 26676 raft_consensus.cc:515] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.101457 26668 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:29.101945 26531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c141b67bfa314acca1c0c3c29d3eaac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.101965 26668 ts_tablet_manager.cc:1403] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.219s	user 0.010s	sys 0.020s
I20260709 12:20:29.104364 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32add6893dda4c028a237f05c1f1ec81" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:29.104252 26667 raft_consensus.cc:359] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.105020 26667 raft_consensus.cc:385] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.105046 26475 raft_consensus.cc:2468] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:29.105412 26667 raft_consensus.cc:740] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.105690 26465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81dffbd4eb274c0595db3619e8eb2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.106196 26667 consensus_queue.cc:260] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.106581 26514 leader_election.cc:304] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.107527 26699 raft_consensus.cc:2804] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.107877 26699 raft_consensus.cc:493] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.108170 26699 raft_consensus.cc:3060] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.110090 26668 raft_consensus.cc:359] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.110809 26668 raft_consensus.cc:385] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.111086 26668 raft_consensus.cc:740] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.112067 26668 consensus_queue.cc:260] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.115823 26667 ts_tablet_manager.cc:1434] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:20:29.116925 26667 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:29.100571 26298 catalog_manager.cc:5697] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:29.121475 26680 raft_consensus.cc:493] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.121958 26680 raft_consensus.cc:515] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.123834 26668 ts_tablet_manager.cc:1434] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:20:29.124577 26680 leader_election.cc:290] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:29.124789 26668 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:29.131592 26667 tablet_bootstrap.cc:654] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.103195 26676 leader_election.cc:290] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:29.136386 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32add6893dda4c028a237f05c1f1ec81" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:29.137105 26399 raft_consensus.cc:2468] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:29.137475 26668 tablet_bootstrap.cc:654] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.136215 26534 tablet_service.cc:1511] Processing CreateTablet for tablet f76e69771f3c45fd842cbfca4e7b66d5 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:29.138957 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f76e69771f3c45fd842cbfca4e7b66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.149065 26670 consensus_queue.cc:260] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.151824 26536 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba667160b44ddf81a0d48eedb7bdae (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:29.153520 26536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba667160b44ddf81a0d48eedb7bdae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.232674 26282 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 85622us
I20260709 12:20:29.239984 26667 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:29.244733 26667 ts_tablet_manager.cc:1403] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.128s	user 0.020s	sys 0.009s
I20260709 12:20:29.245239 26668 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:29.245796 26668 ts_tablet_manager.cc:1403] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.121s	user 0.009s	sys 0.007s
I20260709 12:20:29.248354 26667 raft_consensus.cc:359] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.254473 26670 ts_tablet_manager.cc:1434] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.225s	user 0.006s	sys 0.010s
I20260709 12:20:29.255043 26390 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba667160b44ddf81a0d48eedb7bdae (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:29.257385 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 86ba667160b44ddf81a0d48eedb7bdae (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:29.266124 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba667160b44ddf81a0d48eedb7bdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.267043 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86ba667160b44ddf81a0d48eedb7bdae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.267570 26284 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.16.193.66:37205, user_credentials={real_user=slave}} blocked reactor thread for 114351us
I20260709 12:20:29.260421 26699 raft_consensus.cc:515] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.282920 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32add6893dda4c028a237f05c1f1ec81" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:29.283608 26474 raft_consensus.cc:3060] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.284000 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32add6893dda4c028a237f05c1f1ec81" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:29.284571 26399 raft_consensus.cc:3060] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.260828 26538 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9edfeb05dc4d379a4f87f500ab5f26 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:29.262286 26669 tablet_bootstrap.cc:700] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: Time spent opening tablet: real 0.122s	user 0.001s	sys 0.005s
I20260709 12:20:29.261060 26668 raft_consensus.cc:359] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.263597 26667 raft_consensus.cc:385] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.286615 26668 raft_consensus.cc:385] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.286679 26667 raft_consensus.cc:740] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.287052 26538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9edfeb05dc4d379a4f87f500ab5f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.287024 26668 raft_consensus.cc:740] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.287703 26667 consensus_queue.cc:260] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.288116 26668 consensus_queue.cc:260] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.289955 26667 ts_tablet_manager.cc:1434] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 12:20:29.264741 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df382f80de9947869b3760fc62e30a77" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:29.290735 26668 ts_tablet_manager.cc:1434] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.001s
I20260709 12:20:29.292238 26668 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:29.290961 26475 raft_consensus.cc:2468] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:29.294273 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9edfeb05dc4d379a4f87f500ab5f26 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:29.296916 26670 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:29.306178 26667 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:29.311555 26588 leader_election.cc:304] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.312932 26680 raft_consensus.cc:2804] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.313295 26680 raft_consensus.cc:493] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.313612 26680 raft_consensus.cc:3060] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.294801 26699 leader_election.cc:290] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:29.299925 26474 raft_consensus.cc:2468] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:29.318042 26514 leader_election.cc:304] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.319228 26699 raft_consensus.cc:2804] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.319883 26699 raft_consensus.cc:697] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:29.320950 26699 consensus_queue.cc:237] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.330565 26399 raft_consensus.cc:2468] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:29.332074 26667 tablet_bootstrap.cc:654] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.333535 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9edfeb05dc4d379a4f87f500ab5f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.336772 26680 raft_consensus.cc:515] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.310370 26669 tablet_bootstrap.cc:654] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.299330 26385 tablet_service.cc:1511] Processing CreateTablet for tablet d729d1bc559949d28af8e08f5a719332 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:29.340402 26680 leader_election.cc:290] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:29.346148 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d729d1bc559949d28af8e08f5a719332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.349756 26677 raft_consensus.cc:493] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:29.352142 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:29.351814 26677 raft_consensus.cc:515] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.352406 26605 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9edfeb05dc4d379a4f87f500ab5f26 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:29.352856 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df382f80de9947869b3760fc62e30a77" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:29.353525 26474 raft_consensus.cc:3060] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.353663 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9edfeb05dc4d379a4f87f500ab5f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.361514 26538 tablet_service.cc:1511] Processing CreateTablet for tablet d729d1bc559949d28af8e08f5a719332 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:29.362918 26538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d729d1bc559949d28af8e08f5a719332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.366297 26677 leader_election.cc:290] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:29.367626 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
W20260709 12:20:29.369117 26439 leader_election.cc:343] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.369210 26668 tablet_bootstrap.cc:654] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.377802 26461 tablet_service.cc:1511] Processing CreateTablet for tablet d729d1bc559949d28af8e08f5a719332 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:29.379787 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d729d1bc559949d28af8e08f5a719332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.357139 26680 raft_consensus.cc:493] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.381068 26390 tablet_service.cc:1511] Processing CreateTablet for tablet 966ed0b8df3447849f8bccb9b1af5b33 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:29.383816 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966ed0b8df3447849f8bccb9b1af5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.383724 26680 raft_consensus.cc:515] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.384037 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df382f80de9947869b3760fc62e30a77" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:29.384730 26399 raft_consensus.cc:2468] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:29.385363 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df382f80de9947869b3760fc62e30a77" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:29.385957 26400 raft_consensus.cc:3060] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.387161 26680 leader_election.cc:290] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:29.388303 26465 tablet_service.cc:1511] Processing CreateTablet for tablet 966ed0b8df3447849f8bccb9b1af5b33 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:29.389583 26465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966ed0b8df3447849f8bccb9b1af5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.393429 26386 tablet_service.cc:1511] Processing CreateTablet for tablet d8bad9126db041f7b83ae3e025d0c845 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:29.394891 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bad9126db041f7b83ae3e025d0c845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.381699 26667 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:29.397410 26474 raft_consensus.cc:2468] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:29.398208 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 966ed0b8df3447849f8bccb9b1af5b33 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:29.398435 26606 tablet_service.cc:1511] Processing CreateTablet for tablet d8bad9126db041f7b83ae3e025d0c845 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:29.399542 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 966ed0b8df3447849f8bccb9b1af5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.403646 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bad9126db041f7b83ae3e025d0c845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.406091 26400 raft_consensus.cc:2468] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:29.399555 26588 leader_election.cc:304] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.408280 26680 raft_consensus.cc:2804] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.409351 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:29.411371 26586 leader_election.cc:343] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.415447 26667 ts_tablet_manager.cc:1403] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.015s
I20260709 12:20:29.431612 26680 raft_consensus.cc:697] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:29.418967 26667 raft_consensus.cc:359] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.433054 26667 raft_consensus.cc:385] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.433399 26667 raft_consensus.cc:740] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.434522 26680 consensus_queue.cc:237] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.436321 26699 consensus_queue.cc:1048] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:29.441367 26667 consensus_queue.cc:260] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.461138 26673 raft_consensus.cc:493] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.461325 26611 tablet_service.cc:1511] Processing CreateTablet for tablet a9f838e26a3f48ebb76d7b98527fe2d0 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:29.469470 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f838e26a3f48ebb76d7b98527fe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.457968 26297 catalog_manager.cc:5697] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:29.479118 26465 tablet_service.cc:1511] Processing CreateTablet for tablet a289521cf13242cf80f6cf90c53b8ab4 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:29.472589 26673 raft_consensus.cc:515] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.482092 26465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a289521cf13242cf80f6cf90c53b8ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.488262 26673 leader_election.cc:290] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:29.492538 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa46be07aa5447cad5a356491417c4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:29.492858 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa46be07aa5447cad5a356491417c4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:29.493790 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa46be07aa5447cad5a356491417c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.494578 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa46be07aa5447cad5a356491417c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.495898 26296 catalog_manager.cc:5697] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:29.492537 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:29.500440 26365 leader_election.cc:343] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.507928 26670 tablet_bootstrap.cc:654] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.508749 26669 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:29.509312 26669 ts_tablet_manager.cc:1403] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.449s	user 0.012s	sys 0.007s
W20260709 12:20:29.525182 26439 consensus_peers.cc:597] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e -> Peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Couldn't send request to peer 39930cd9790d4943bdec6f9df4ce2641. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:29.524950 26461 tablet_service.cc:1511] Processing CreateTablet for tablet a9f838e26a3f48ebb76d7b98527fe2d0 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:20:29.534245 26514 consensus_peers.cc:597] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:29.534330 26531 tablet_service.cc:1511] Processing CreateTablet for tablet d8bad9126db041f7b83ae3e025d0c845 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:29.535836 26531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8bad9126db041f7b83ae3e025d0c845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.535646 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 4aa46be07aa5447cad5a356491417c4c (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:29.537014 26532 tablet_service.cc:1511] Processing CreateTablet for tablet a9f838e26a3f48ebb76d7b98527fe2d0 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:29.538244 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f838e26a3f48ebb76d7b98527fe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.542881 26538 tablet_service.cc:1511] Processing CreateTablet for tablet a289521cf13242cf80f6cf90c53b8ab4 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:29.527102 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f838e26a3f48ebb76d7b98527fe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.544106 26538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a289521cf13242cf80f6cf90c53b8ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.547977 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aa46be07aa5447cad5a356491417c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.555670 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 52d9fdb0a26d4b43a1940f3ca15c530f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:29.557298 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d9fdb0a26d4b43a1940f3ca15c530f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.560319 26668 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:29.560849 26668 ts_tablet_manager.cc:1403] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.269s	user 0.022s	sys 0.022s
I20260709 12:20:29.564194 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 52d9fdb0a26d4b43a1940f3ca15c530f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:29.564528 26463 tablet_service.cc:1511] Processing CreateTablet for tablet 050c5f3e50244a45886a0ee8245165af (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:29.565500 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d9fdb0a26d4b43a1940f3ca15c530f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.565845 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050c5f3e50244a45886a0ee8245165af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.568063 26668 raft_consensus.cc:359] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.568848 26668 raft_consensus.cc:385] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.571147 26669 raft_consensus.cc:359] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.571898 26669 raft_consensus.cc:385] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.572146 26669 raft_consensus.cc:740] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.572784 26669 consensus_queue.cc:260] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.580799 26669 ts_tablet_manager.cc:1434] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.003s
I20260709 12:20:29.581197 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:29.581823 26549 raft_consensus.cc:2468] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:29.583298 26589 leader_election.cc:304] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:29.584403 26680 raft_consensus.cc:2804] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.584774 26680 raft_consensus.cc:493] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.585090 26680 raft_consensus.cc:3060] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.569152 26668 raft_consensus.cc:740] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.603647 26667 ts_tablet_manager.cc:1434] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.188s	user 0.007s	sys 0.012s
I20260709 12:20:29.604882 26667 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:29.607686 26670 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:29.608260 26670 ts_tablet_manager.cc:1403] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.312s	user 0.001s	sys 0.016s
I20260709 12:20:29.608536 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:29.608906 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:29.611004 26441 leader_election.cc:343] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.611505 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 050c5f3e50244a45886a0ee8245165af (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:29.611495 26539 tablet_service.cc:1511] Processing CreateTablet for tablet 52d9fdb0a26d4b43a1940f3ca15c530f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:29.613157 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050c5f3e50244a45886a0ee8245165af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.613020 26366 leader_election.cc:343] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.570125 26385 tablet_service.cc:1511] Processing CreateTablet for tablet a289521cf13242cf80f6cf90c53b8ab4 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:29.613781 26366 leader_election.cc:304] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:29.614930 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a289521cf13242cf80f6cf90c53b8ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.611545 26441 leader_election.cc:304] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:29.615494 26722 raft_consensus.cc:2749] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:29.616556 26677 raft_consensus.cc:2749] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:29.617904 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d9fdb0a26d4b43a1940f3ca15c530f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.601610 26680 raft_consensus.cc:515] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.621634 26680 leader_election.cc:290] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:29.622838 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:29.623796 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:29.624398 26586 leader_election.cc:343] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.624686 26549 raft_consensus.cc:3060] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.626158 26670 raft_consensus.cc:359] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.627409 26668 consensus_queue.cc:260] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.629827 26668 ts_tablet_manager.cc:1434] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:20:29.630311 26669 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:29.637485 26669 tablet_bootstrap.cc:654] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.626910 26670 raft_consensus.cc:385] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.643611 26670 raft_consensus.cc:740] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.643952 26667 tablet_bootstrap.cc:654] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.646230 26670 consensus_queue.cc:260] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:29.648762 26670 ts_tablet_manager.cc:1434] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 12:20:29.649827 26670 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:29.652451 26386 tablet_service.cc:1511] Processing CreateTablet for tablet a97fd66be7ff4a588ce7be8aa1a5b5be (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:29.654086 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fd66be7ff4a588ce7be8aa1a5b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.656042 26668 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:29.664826 26677 consensus_queue.cc:1048] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:29.669521 26668 tablet_bootstrap.cc:654] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.671587 26457 tablet_service.cc:1511] Processing CreateTablet for tablet a97fd66be7ff4a588ce7be8aa1a5b5be (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:29.673352 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fd66be7ff4a588ce7be8aa1a5b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.682421 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 050c5f3e50244a45886a0ee8245165af (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:29.683959 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 050c5f3e50244a45886a0ee8245165af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.697319 26532 tablet_service.cc:1511] Processing CreateTablet for tablet a97fd66be7ff4a588ce7be8aa1a5b5be (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:29.697400 26613 tablet_service.cc:1511] Processing CreateTablet for tablet d7cc64cd1a8348f68f5b9115f9205e40 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:29.697340 26385 tablet_service.cc:1511] Processing CreateTablet for tablet d7cc64cd1a8348f68f5b9115f9205e40 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:29.698761 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cc64cd1a8348f68f5b9115f9205e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.699409 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cc64cd1a8348f68f5b9115f9205e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.712054 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97fd66be7ff4a588ce7be8aa1a5b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.714154 26668 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:29.714708 26668 ts_tablet_manager.cc:1403] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.003s
I20260709 12:20:29.717926 26668 raft_consensus.cc:359] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.718684 26668 raft_consensus.cc:385] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.722298 26535 tablet_service.cc:1511] Processing CreateTablet for tablet c6ee0ada749043888b2a134a520a25e1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:29.723625 26535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ee0ada749043888b2a134a520a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.731037 26463 tablet_service.cc:1511] Processing CreateTablet for tablet d7cc64cd1a8348f68f5b9115f9205e40 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:29.718990 26668 raft_consensus.cc:740] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.737254 26549 raft_consensus.cc:2468] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:29.744669 26589 leader_election.cc:304] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:29.745961 26721 raft_consensus.cc:2804] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.746464 26721 raft_consensus.cc:697] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:29.748958 26721 consensus_queue.cc:237] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.739543 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cc64cd1a8348f68f5b9115f9205e40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.740499 26669 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:29.742915 26667 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:29.753417 26669 ts_tablet_manager.cc:1403] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.017s
I20260709 12:20:29.753557 26667 ts_tablet_manager.cc:1403] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.012s
I20260709 12:20:29.735198 26668 consensus_queue.cc:260] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.757063 26668 ts_tablet_manager.cc:1434] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.042s	user 0.000s	sys 0.007s
I20260709 12:20:29.757496 26676 consensus_queue.cc:1048] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:29.758153 26668 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:29.759027 26670 tablet_bootstrap.cc:654] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.756448 26669 raft_consensus.cc:359] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.760828 26669 raft_consensus.cc:385] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.761101 26669 raft_consensus.cc:740] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.761951 26669 consensus_queue.cc:260] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.761982 26667 raft_consensus.cc:359] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.762908 26667 raft_consensus.cc:385] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.763221 26667 raft_consensus.cc:740] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.764217 26669 ts_tablet_manager.cc:1434] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:20:29.778447 26669 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:29.778192 26667 consensus_queue.cc:260] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.780786 26298 catalog_manager.cc:5697] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:29.790350 26535 tablet_service.cc:1511] Processing CreateTablet for tablet 696254bba321492ab3ca55d7648e313b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:29.791565 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 696254bba321492ab3ca55d7648e313b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:29.792187 26535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696254bba321492ab3ca55d7648e313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.792752 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696254bba321492ab3ca55d7648e313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.765895 26613 tablet_service.cc:1511] Processing CreateTablet for tablet c6ee0ada749043888b2a134a520a25e1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:29.795625 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ee0ada749043888b2a134a520a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.797545 26461 tablet_service.cc:1511] Processing CreateTablet for tablet 696254bba321492ab3ca55d7648e313b (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:29.798731 26386 tablet_service.cc:1511] Processing CreateTablet for tablet c6ee0ada749043888b2a134a520a25e1 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:29.800287 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ee0ada749043888b2a134a520a25e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.802119 26385 tablet_service.cc:1511] Processing CreateTablet for tablet 235d0175b299499399baaa177b25de35 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:29.803110 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 235d0175b299499399baaa177b25de35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.809556 26465 tablet_service.cc:1511] Processing CreateTablet for tablet 235d0175b299499399baaa177b25de35 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:29.825456 26669 tablet_bootstrap.cc:654] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.827419 26667 ts_tablet_manager.cc:1434] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 12:20:29.828450 26461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 696254bba321492ab3ca55d7648e313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.830289 26465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 235d0175b299499399baaa177b25de35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.831422 26605 tablet_service.cc:1511] Processing CreateTablet for tablet 948eb78b01d544e6a0e64f60b261edea (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:29.832818 26605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948eb78b01d544e6a0e64f60b261edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.835215 26539 tablet_service.cc:1511] Processing CreateTablet for tablet 235d0175b299499399baaa177b25de35 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:29.836508 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 235d0175b299499399baaa177b25de35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.838410 26670 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:29.838930 26670 ts_tablet_manager.cc:1403] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.189s	user 0.007s	sys 0.007s
I20260709 12:20:29.842338 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 948eb78b01d544e6a0e64f60b261edea (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:29.846526 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948eb78b01d544e6a0e64f60b261edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.847884 26668 tablet_bootstrap.cc:654] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.847565 26670 raft_consensus.cc:359] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.848379 26670 raft_consensus.cc:385] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:29.848712 26670 raft_consensus.cc:740] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:29.849455 26670 consensus_queue.cc:260] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.853547 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6c90924ba74445ae2edc110c400e78 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:29.854938 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6c90924ba74445ae2edc110c400e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.861382 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 948eb78b01d544e6a0e64f60b261edea (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:29.862588 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948eb78b01d544e6a0e64f60b261edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.864321 26390 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6c90924ba74445ae2edc110c400e78 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:29.865679 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6c90924ba74445ae2edc110c400e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.866585 26458 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcf2bebeaf8451e890bcbe46a6ae967 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:29.867931 26458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcf2bebeaf8451e890bcbe46a6ae967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.871948 26734 raft_consensus.cc:493] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.872687 26734 raft_consensus.cc:515] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.878293 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:29.878981 26549 raft_consensus.cc:2468] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:29.880350 26607 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcf2bebeaf8451e890bcbe46a6ae967 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:29.880918 26532 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcf2bebeaf8451e890bcbe46a6ae967 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:29.881784 26607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcf2bebeaf8451e890bcbe46a6ae967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.874787 26670 ts_tablet_manager.cc:1434] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.003s
I20260709 12:20:29.882417 26532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcf2bebeaf8451e890bcbe46a6ae967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.883956 26441 leader_election.cc:304] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.884877 26731 raft_consensus.cc:493] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.885324 26731 raft_consensus.cc:515] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.887410 26677 raft_consensus.cc:2804] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.887804 26677 raft_consensus.cc:493] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.888149 26677 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.892401 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:29.892972 26731 leader_election.cc:290] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:29.893103 26400 raft_consensus.cc:2468] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:29.894670 26670 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:29.894580 26438 leader_election.cc:304] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:29.895525 26731 raft_consensus.cc:2804] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:29.895893 26731 raft_consensus.cc:493] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:29.896205 26731 raft_consensus.cc:3060] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.902318 26699 consensus_queue.cc:1048] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:29.874975 26667 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:29.903169 26731 raft_consensus.cc:515] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:29.920915 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:29.932025 26389 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc216f2a251463a9c406ab418b62360 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:29.919756 26734 leader_election.cc:290] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:29.907282 26460 tablet_service.cc:1511] Processing CreateTablet for tablet ad43a300a529407993bd1819f2e09aaa (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:29.933683 26389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc216f2a251463a9c406ab418b62360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.934506 26460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad43a300a529407993bd1819f2e09aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.903888 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 9f6c90924ba74445ae2edc110c400e78 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 12:20:29.933614 26439 leader_election.cc:343] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.936470 26677 raft_consensus.cc:515] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.937023 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:29.937633 26400 raft_consensus.cc:3060] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.939284 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:29.939929 26548 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:29.941985 26670 tablet_bootstrap.cc:654] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:29.907861 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:29.905891 26385 tablet_service.cc:1511] Processing CreateTablet for tablet ad43a300a529407993bd1819f2e09aaa (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:29.944408 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad43a300a529407993bd1819f2e09aaa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:29.944347 26441 leader_election.cc:343] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:29.927727 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc216f2a251463a9c406ab418b62360 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:29.936175 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f6c90924ba74445ae2edc110c400e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.947288 26539 tablet_service.cc:1511] Processing CreateTablet for tablet ad43a300a529407993bd1819f2e09aaa (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:29.948560 26539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad43a300a529407993bd1819f2e09aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.951543 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc216f2a251463a9c406ab418b62360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.951898 26548 raft_consensus.cc:2468] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:29.953085 26441 leader_election.cc:304] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:29.951167 26463 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc216f2a251463a9c406ab418b62360 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:29.954521 26463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc216f2a251463a9c406ab418b62360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.956368 26734 raft_consensus.cc:2804] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:29.957895 26734 raft_consensus.cc:697] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:29.958717 26734 consensus_queue.cc:237] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:29.961127 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 3bbbf751e4544950af8ef067e8a1f70f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:29.962299 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bbbf751e4544950af8ef067e8a1f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:29.928874 26668 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:29.965188 26722 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.965672 26722 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:29.974160 26722 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:29.976248 26677 leader_election.cc:290] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:29.981436 26668 ts_tablet_manager.cc:1403] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.219s	user 0.004s	sys 0.010s
I20260709 12:20:29.990273 26731 leader_election.cc:290] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:29.994663 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:30.015488 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 3bbbf751e4544950af8ef067e8a1f70f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:30.017082 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bbbf751e4544950af8ef067e8a1f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.019055 26400 raft_consensus.cc:2468] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:30.025947 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 953f66c8075b448fa1ceb677489c5129 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:30.027544 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953f66c8075b448fa1ceb677489c5129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.014067 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.033993 26548 raft_consensus.cc:2468] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:30.052753 26366 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.054270 26669 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:30.054783 26669 ts_tablet_manager.cc:1403] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.277s	user 0.020s	sys 0.011s
I20260709 12:20:30.055413 26722 raft_consensus.cc:2804] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.055825 26722 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.056138 26722 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:30.047165 26439 leader_election.cc:343] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.062727 26534 tablet_service.cc:1511] Processing CreateTablet for tablet 953f66c8075b448fa1ceb677489c5129 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:30.065120 26721 consensus_queue.cc:1048] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.065546 26622 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.065884 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953f66c8075b448fa1ceb677489c5129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.052326 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:30.076902 26438 leader_election.cc:304] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.077791 26734 raft_consensus.cc:2804] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.078279 26734 raft_consensus.cc:697] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:30.079682 26385 tablet_service.cc:1511] Processing CreateTablet for tablet b91ba46bcd3e4802a197552bcbda449f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:30.080935 26385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b91ba46bcd3e4802a197552bcbda449f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.079106 26734 consensus_queue.cc:237] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:30.087571 26441 leader_election.cc:343] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.089555 26457 tablet_service.cc:1511] Processing CreateTablet for tablet b91ba46bcd3e4802a197552bcbda449f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:30.090409 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b91ba46bcd3e4802a197552bcbda449f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.096735 26669 raft_consensus.cc:359] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.097476 26669 raft_consensus.cc:385] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.097322 26668 raft_consensus.cc:359] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.097863 26669 raft_consensus.cc:740] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.100047 26669 consensus_queue.cc:260] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.079396 26390 tablet_service.cc:1511] Processing CreateTablet for tablet 3bbbf751e4544950af8ef067e8a1f70f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:30.101954 26390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bbbf751e4544950af8ef067e8a1f70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.102725 26669 ts_tablet_manager.cc:1434] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.048s	user 0.003s	sys 0.006s
I20260709 12:20:30.090754 26389 tablet_service.cc:1511] Processing CreateTablet for tablet 063cdd050c3d40fdab471dec902e20e7 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:30.107146 26535 tablet_service.cc:1511] Processing CreateTablet for tablet 095ead81e4364b36a759035a2def0683 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:30.108269 26389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063cdd050c3d40fdab471dec902e20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.109252 26535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095ead81e4364b36a759035a2def0683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.113061 26667 tablet_bootstrap.cc:654] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.118155 26386 tablet_service.cc:1511] Processing CreateTablet for tablet 095ead81e4364b36a759035a2def0683 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:30.118366 26457 tablet_service.cc:1511] Processing CreateTablet for tablet 063cdd050c3d40fdab471dec902e20e7 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:30.119256 26386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095ead81e4364b36a759035a2def0683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.098166 26668 raft_consensus.cc:385] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.126405 26669 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:30.130322 26722 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.119292 26457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063cdd050c3d40fdab471dec902e20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.133579 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:30.134991 26548 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.124326 26668 raft_consensus.cc:740] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.141781 26534 tablet_service.cc:1511] Processing CreateTablet for tablet b91ba46bcd3e4802a197552bcbda449f (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:30.143365 26534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b91ba46bcd3e4802a197552bcbda449f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.147790 26668 consensus_queue.cc:260] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.155695 26670 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.156522 26670 ts_tablet_manager.cc:1403] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.262s	user 0.014s	sys 0.005s
I20260709 12:20:30.151101 26668 ts_tablet_manager.cc:1434] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.156s	user 0.004s	sys 0.003s
I20260709 12:20:30.159569 26668 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:30.160753 26670 raft_consensus.cc:359] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.161552 26670 raft_consensus.cc:385] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.161870 26670 raft_consensus.cc:740] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.163092 26680 raft_consensus.cc:493] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.163815 26680 raft_consensus.cc:515] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.166033 26680 leader_election.cc:290] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:30.175649 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.176496 26548 raft_consensus.cc:2468] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:30.181020 26613 tablet_service.cc:1511] Processing CreateTablet for tablet 063cdd050c3d40fdab471dec902e20e7 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:30.182474 26613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063cdd050c3d40fdab471dec902e20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.182663 26366 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.182816 26680 consensus_queue.cc:1048] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.180967 26611 tablet_service.cc:1511] Processing CreateTablet for tablet 953f66c8075b448fa1ceb677489c5129 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:30.185516 26611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953f66c8075b448fa1ceb677489c5129. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:30.177145 26589 leader_election.cc:343] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.182266 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.181226 26606 tablet_service.cc:1511] Processing CreateTablet for tablet 095ead81e4364b36a759035a2def0683 (DEFAULT_TABLE table=test-workload [id=bf0a8b0ff9c349a08a6b9e66e128e93f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:30.141286 26722 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:30.185541 26669 tablet_bootstrap.cc:654] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.203981 26606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095ead81e4364b36a759035a2def0683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:30.206157 26722 raft_consensus.cc:2804] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.206712 26722 raft_consensus.cc:697] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:30.208110 26722 consensus_queue.cc:237] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.222718 26298 catalog_manager.cc:5697] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.224452 26298 catalog_manager.cc:5697] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.212175 26670 consensus_queue.cc:260] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.229791 26670 ts_tablet_manager.cc:1434] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.001s
I20260709 12:20:30.230950 26670 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.232424 26668 tablet_bootstrap.cc:654] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:30.234105 26366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 51432.7us
W20260709 12:20:30.240494 26586 consensus_peers.cc:597] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:30.241168 26586 leader_election.cc:343] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.241616 26586 leader_election.cc:304] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:30.244830 26670 tablet_bootstrap.cc:654] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.245923 26680 raft_consensus.cc:2749] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.267581 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:30.268462 26623 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.266431 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:30.288149 26667 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.288646 26669 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:30.294796 26669 ts_tablet_manager.cc:1403] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.169s	user 0.017s	sys 0.001s
I20260709 12:20:30.295074 26667 ts_tablet_manager.cc:1403] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.423s	user 0.016s	sys 0.003s
I20260709 12:20:30.298100 26623 raft_consensus.cc:2468] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:30.300374 26667 raft_consensus.cc:359] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.301026 26667 raft_consensus.cc:385] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.301342 26667 raft_consensus.cc:740] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.301728 26669 raft_consensus.cc:359] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.302464 26669 raft_consensus.cc:385] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.302752 26669 raft_consensus.cc:740] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.303512 26669 consensus_queue.cc:260] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.304355 26296 catalog_manager.cc:5697] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.305753 26669 ts_tablet_manager.cc:1434] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:20:30.307121 26669 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:30.307281 26667 consensus_queue.cc:260] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.310552 26667 ts_tablet_manager.cc:1434] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:20:30.311379 26668 tablet_bootstrap.cc:492] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:30.311611 26667 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.311864 26668 ts_tablet_manager.cc:1403] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.153s	user 0.002s	sys 0.015s
I20260709 12:20:30.318486 26667 tablet_bootstrap.cc:654] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.320724 26670 tablet_bootstrap.cc:492] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.321411 26670 ts_tablet_manager.cc:1403] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.091s	user 0.024s	sys 0.026s
I20260709 12:20:30.324128 26680 consensus_queue.cc:1048] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.326189 26670 raft_consensus.cc:359] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.326990 26670 raft_consensus.cc:385] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.327374 26670 raft_consensus.cc:740] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.328267 26669 tablet_bootstrap.cc:654] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.328264 26670 consensus_queue.cc:260] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.331058 26667 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.331058 26670 ts_tablet_manager.cc:1434] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.006s
I20260709 12:20:30.331732 26667 ts_tablet_manager.cc:1403] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 12:20:30.332683 26670 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.331346 26668 raft_consensus.cc:359] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.333292 26668 raft_consensus.cc:385] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.333585 26668 raft_consensus.cc:740] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.334501 26668 consensus_queue.cc:260] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.334739 26667 raft_consensus.cc:359] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.335589 26667 raft_consensus.cc:385] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.335886 26667 raft_consensus.cc:740] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.336665 26667 consensus_queue.cc:260] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.340965 26667 ts_tablet_manager.cc:1434] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:30.341241 26670 tablet_bootstrap.cc:654] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.341882 26667 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.349021 26667 tablet_bootstrap.cc:654] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.353510 26668 ts_tablet_manager.cc:1434] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
W20260709 12:20:30.354882 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:30.358680 26670 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.359218 26670 ts_tablet_manager.cc:1403] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.013s
I20260709 12:20:30.359979 26668 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:30.369522 26667 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.370078 26667 ts_tablet_manager.cc:1403] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.003s
W20260709 12:20:30.370863 26439 consensus_peers.cc:597] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e -> Peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Couldn't send request to peer 39930cd9790d4943bdec6f9df4ce2641. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:30.362053 26670 raft_consensus.cc:359] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.371407 26670 raft_consensus.cc:385] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.371716 26670 raft_consensus.cc:740] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.372402 26670 consensus_queue.cc:260] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.372983 26667 raft_consensus.cc:359] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.373889 26667 raft_consensus.cc:385] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.374212 26667 raft_consensus.cc:740] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.374532 26670 ts_tablet_manager.cc:1434] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.001s
I20260709 12:20:30.375109 26667 consensus_queue.cc:260] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.376775 26670 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.377491 26667 ts_tablet_manager.cc:1434] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:20:30.378511 26667 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.382884 26670 tablet_bootstrap.cc:654] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.384632 26668 tablet_bootstrap.cc:654] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.391645 26667 tablet_bootstrap.cc:654] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.398090 26669 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:30.398626 26667 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.398698 26669 ts_tablet_manager.cc:1403] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.018s
I20260709 12:20:30.399269 26667 ts_tablet_manager.cc:1403] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 12:20:30.400079 26670 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.400537 26670 ts_tablet_manager.cc:1403] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.004s
I20260709 12:20:30.402132 26667 raft_consensus.cc:359] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.402814 26667 raft_consensus.cc:385] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.403391 26667 raft_consensus.cc:740] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.402725 26669 raft_consensus.cc:359] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.403999 26669 raft_consensus.cc:385] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.404422 26669 raft_consensus.cc:740] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.404386 26667 consensus_queue.cc:260] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.406409 26668 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:30.405753 26669 consensus_queue.cc:260] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.406919 26668 ts_tablet_manager.cc:1403] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.007s
I20260709 12:20:30.409533 26722 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.410065 26722 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.413486 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.414149 26722 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.414386 26680 raft_consensus.cc:493] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.402995 26670 raft_consensus.cc:359] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.414786 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.415045 26680 raft_consensus.cc:515] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.416373 26667 ts_tablet_manager.cc:1434] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:20:30.416443 26548 raft_consensus.cc:2468] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
W20260709 12:20:30.416790 26365 leader_election.cc:343] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.413554 26668 raft_consensus.cc:359] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.417433 26668 raft_consensus.cc:385] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.417717 26668 raft_consensus.cc:740] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.418524 26668 consensus_queue.cc:260] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.419309 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.419994 26475 raft_consensus.cc:2468] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:30.420967 26667 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.421756 26668 ts_tablet_manager.cc:1434] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:20:30.420008 26366 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:30.424798 26722 raft_consensus.cc:2804] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.424952 26669 ts_tablet_manager.cc:1434] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.005s
I20260709 12:20:30.425199 26722 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.425642 26722 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.426084 26669 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:30.428987 26667 tablet_bootstrap.cc:654] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.432420 26588 leader_election.cc:304] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:30.433440 26668 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:30.434245 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.433398 26721 raft_consensus.cc:2804] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.435519 26721 raft_consensus.cc:493] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.435992 26721 raft_consensus.cc:3060] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.439330 26669 tablet_bootstrap.cc:654] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.443702 26668 tablet_bootstrap.cc:654] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.451817 26680 leader_election.cc:290] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.435129 26548 raft_consensus.cc:2468] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:30.465921 26475 raft_consensus.cc:3060] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.475978 26669 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:30.476616 26669 ts_tablet_manager.cc:1403] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.010s
I20260709 12:20:30.415045 26670 raft_consensus.cc:385] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.479635 26670 raft_consensus.cc:740] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.479729 26669 raft_consensus.cc:359] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.480594 26670 consensus_queue.cc:260] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.481518 26722 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.480614 26669 raft_consensus.cc:385] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.483335 26669 raft_consensus.cc:740] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.485523 26669 consensus_queue.cc:260] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.485689 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:30.487787 26731 consensus_queue.cc:1048] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:30.489182 26365 leader_election.cc:343] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.489564 26669 ts_tablet_manager.cc:1434] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 12:20:30.491874 26731 raft_consensus.cc:493] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.492409 26731 raft_consensus.cc:515] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.494982 26731 leader_election.cc:290] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.495167 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.495872 26399 raft_consensus.cc:2468] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:30.497149 26438 leader_election.cc:304] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.498014 26731 raft_consensus.cc:2804] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.498383 26731 raft_consensus.cc:493] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.498747 26731 raft_consensus.cc:3060] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.500361 26670 ts_tablet_manager.cc:1434] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.100s	user 0.006s	sys 0.002s
I20260709 12:20:30.501332 26670 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.507393 26721 raft_consensus.cc:515] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.510092 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:30.511917 26441 leader_election.cc:343] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.513728 26669 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:30.511700 26731 raft_consensus.cc:515] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.517550 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:30.517835 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:30.518178 26399 raft_consensus.cc:3060] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.518802 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:30.519454 26474 raft_consensus.cc:3060] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:30.519578 26441 leader_election.cc:343] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.520361 26721 leader_election.cc:290] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.520569 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9607fece157456e9d07bf4ef22d370c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:30.521212 26549 raft_consensus.cc:3060] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.521677 26722 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.523098 26670 tablet_bootstrap.cc:654] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.523131 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:30.524053 26547 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.526898 26669 tablet_bootstrap.cc:654] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.527661 26399 raft_consensus.cc:2468] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:30.528903 26438 leader_election.cc:304] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:30.529770 26734 raft_consensus.cc:2804] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.530313 26734 raft_consensus.cc:697] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:30.531198 26734 consensus_queue.cc:237] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.534833 26731 leader_election.cc:290] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.533718 26474 raft_consensus.cc:2468] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:30.545351 26588 leader_election.cc:304] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:30.546468 26721 raft_consensus.cc:2804] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.547122 26721 raft_consensus.cc:697] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:30.552806 26668 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:30.553254 26668 ts_tablet_manager.cc:1403] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.120s	user 0.021s	sys 0.016s
I20260709 12:20:30.548908 26547 raft_consensus.cc:2468] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:30.549429 26549 raft_consensus.cc:2468] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:30.555831 26764 raft_consensus.cc:493] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.561673 26740 consensus_queue.cc:1048] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.561849 26764 raft_consensus.cc:515] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.564277 26764 leader_election.cc:290] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.564985 26546 raft_consensus.cc:3060] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.548208 26721 consensus_queue.cc:237] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.570377 26722 raft_consensus.cc:493] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.570626 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.571563 26722 raft_consensus.cc:515] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.573819 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759e0f60b4604c2e9abda3833a0e9e6f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.576735 26668 raft_consensus.cc:359] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.577667 26668 raft_consensus.cc:385] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.578013 26668 raft_consensus.cc:740] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.580721 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:30.580560 26668 consensus_queue.cc:260] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.573854 26722 leader_election.cc:290] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:30.580741 26514 leader_election.cc:304] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:30.582604 26668 ts_tablet_manager.cc:1434] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:20:30.583679 26668 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:30.581807 26366 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:30.587091 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85165efa340d47638ef75f49dda75e8a" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.586035 26722 raft_consensus.cc:2804] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.589087 26722 raft_consensus.cc:697] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:30.590296 26722 consensus_queue.cc:237] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.597627 26366 leader_election.cc:304] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:30.598834 26750 raft_consensus.cc:2749] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.601258 26667 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.600733 26668 tablet_bootstrap.cc:654] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.601778 26667 ts_tablet_manager.cc:1403] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.181s	user 0.038s	sys 0.051s
I20260709 12:20:30.603556 26670 tablet_bootstrap.cc:492] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.604090 26670 ts_tablet_manager.cc:1403] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.000s
I20260709 12:20:30.607533 26750 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.608067 26750 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.609459 26670 raft_consensus.cc:359] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.610423 26670 raft_consensus.cc:385] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.610778 26670 raft_consensus.cc:740] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.614193 26670 consensus_queue.cc:260] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.617072 26670 ts_tablet_manager.cc:1434] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:20:30.617972 26670 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.619711 26731 consensus_queue.cc:1048] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:30.623520 26764 raft_consensus.cc:2749] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.627216 26667 raft_consensus.cc:359] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.628937 26667 raft_consensus.cc:385] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.629218 26667 raft_consensus.cc:740] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.624557 26699 raft_consensus.cc:493] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.631784 26667 consensus_queue.cc:260] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.649674 26669 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:30.638935 26721 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.650241 26669 ts_tablet_manager.cc:1403] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.013s
I20260709 12:20:30.652343 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:30.655354 26740 raft_consensus.cc:493] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.655869 26740 raft_consensus.cc:515] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.656739 26670 tablet_bootstrap.cc:654] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.658143 26740 leader_election.cc:290] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:30.632427 26699 raft_consensus.cc:515] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.656553 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.660015 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe3aff16fd4242f5a9ef1a3038bbe22f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.651842 26750 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.661159 26721 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.662449 26474 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:30.663846 26365 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.666489 26298 catalog_manager.cc:5697] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.661810 26399 raft_consensus.cc:2468] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:30.678275 26721 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.686992 26668 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:30.687552 26668 ts_tablet_manager.cc:1403] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.022s
I20260709 12:20:30.687922 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.688654 26475 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:30.690769 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:30.691455 26621 raft_consensus.cc:2468] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:30.690439 26668 raft_consensus.cc:359] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.693048 26668 raft_consensus.cc:385] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.692961 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe3aff16fd4242f5a9ef1a3038bbe22f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:30.693508 26513 leader_election.cc:304] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:30.693650 26668 raft_consensus.cc:740] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.694087 26622 raft_consensus.cc:2468] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:30.694626 26764 raft_consensus.cc:2804] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.695088 26764 raft_consensus.cc:493] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.694828 26668 consensus_queue.cc:260] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.674645 26750 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.695545 26764 raft_consensus.cc:3060] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.675446 26623 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:30.697000 26750 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.698681 26588 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:30.700799 26750 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.701380 26680 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.701656 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.701753 26680 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.687621 26438 leader_election.cc:304] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.702396 26680 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.703488 26740 raft_consensus.cc:2804] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:30.703845 26740 raft_consensus.cc:493] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.704159 26740 raft_consensus.cc:3060] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:30.704698 26775 log.cc:927] Time spent T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:30.701944 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.707577 26400 raft_consensus.cc:2468] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:30.709028 26668 ts_tablet_manager.cc:1434] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:20:30.709817 26668 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:30.723434 26778 consensus_queue.cc:1048] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:20:30.725764 26669 raft_consensus.cc:359] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.726994 26669 raft_consensus.cc:385] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.727782 26669 raft_consensus.cc:740] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.728513 26669 consensus_queue.cc:260] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.731575 26764 raft_consensus.cc:515] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.733137 26294 catalog_manager.cc:5697] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:30.740415 26669 ts_tablet_manager.cc:1434] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.090s	user 0.003s	sys 0.006s
I20260709 12:20:30.737251 26740 raft_consensus.cc:515] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.743250 26669 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:30.748792 26766 raft_consensus.cc:493] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.749373 26699 leader_election.cc:290] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:30.749938 26667 ts_tablet_manager.cc:1434] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.148s	user 0.014s	sys 0.000s
I20260709 12:20:30.749816 26731 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.749612 26750 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.734308 26764 leader_election.cc:290] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:30.756927 26667 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.762822 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:30.770561 26400 raft_consensus.cc:3060] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.771502 26750 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.771886 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:30.777137 26399 raft_consensus.cc:2393] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:30.757876 26766 raft_consensus.cc:515] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.784148 26766 leader_election.cc:290] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.765970 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97b53680d1349fa8b2bb3c5e067602d" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:30.758986 26731 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:30.785043 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.791884 26623 raft_consensus.cc:3060] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.779628 26680 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:30.804145 26670 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:30.804657 26670 ts_tablet_manager.cc:1403] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.187s	user 0.012s	sys 0.022s
I20260709 12:20:30.809154 26670 raft_consensus.cc:359] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.814106 26670 raft_consensus.cc:385] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:30.814368 26670 raft_consensus.cc:740] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.812599 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:30.815570 26670 consensus_queue.cc:260] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:30.823503 26397 raft_consensus.cc:3060] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.825902 26622 raft_consensus.cc:2393] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:30.820294 26398 raft_consensus.cc:2393] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:30.829717 26782 consensus_queue.cc:1048] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:30.813834 26731 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.838284 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f22e34b989c46e1867151870954a0e5" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.844153 26667 tablet_bootstrap.cc:654] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.844779 26670 ts_tablet_manager.cc:1434] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 12:20:30.845849 26670 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:30.848127 26298 catalog_manager.cc:5697] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.850292 26400 raft_consensus.cc:2468] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:30.851825 26512 leader_election.cc:304] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:30.853452 26784 raft_consensus.cc:2804] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.854056 26784 raft_consensus.cc:697] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:30.855084 26784 consensus_queue.cc:237] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.856478 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:30.857075 26475 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.860455 26514 leader_election.cc:304] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:30.861775 26699 raft_consensus.cc:3060] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.865551 26680 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:30.866472 26680 raft_consensus.cc:493] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.866916 26680 raft_consensus.cc:515] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:30.870324 26785 log.cc:927] Time spent T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:20:30.872761 26740 leader_election.cc:290] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:30.874462 26623 raft_consensus.cc:2468] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:30.880386 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:30.885591 26400 raft_consensus.cc:2393] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:30.912768 26668 tablet_bootstrap.cc:654] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:30.910768 26441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 126739us
I20260709 12:20:30.916401 26670 tablet_bootstrap.cc:654] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.919049 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.930729 26669 tablet_bootstrap.cc:654] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:30.932416 26549 raft_consensus.cc:2393] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:30.948300 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:30.953752 26441 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:30.944067 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe3aff16fd4242f5a9ef1a3038bbe22f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:30.951098 26680 leader_election.cc:290] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:30.958253 26734 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.960896 26400 raft_consensus.cc:3060] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.963802 26667 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:30.964329 26667 ts_tablet_manager.cc:1403] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.208s	user 0.016s	sys 0.006s
I20260709 12:20:30.965530 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:30.965945 26734 raft_consensus.cc:2749] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.967326 26667 raft_consensus.cc:359] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.968036 26667 raft_consensus.cc:385] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:30.967250 26589 leader_election.cc:343] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:30.968309 26667 raft_consensus.cc:740] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:30.969276 26667 consensus_queue.cc:260] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:30.970728 26699 raft_consensus.cc:2749] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.971722 26667 ts_tablet_manager.cc:1434] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:30.972790 26667 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:30.979945 26475 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:30.982818 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe3aff16fd4242f5a9ef1a3038bbe22f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:30.983652 26623 raft_consensus.cc:3060] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.972152 26400 raft_consensus.cc:2468] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:30.989063 26295 catalog_manager.cc:5697] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:30.994154 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:30.994413 26622 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:30.995479 26365 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:30.996030 26668 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:30.996562 26668 ts_tablet_manager.cc:1403] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.287s	user 0.006s	sys 0.010s
I20260709 12:20:30.999439 26722 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:30.999354 26668 raft_consensus.cc:359] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.000001 26722 raft_consensus.cc:697] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:31.000490 26667 tablet_bootstrap.cc:654] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.000877 26722 consensus_queue.cc:237] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.001680 26623 raft_consensus.cc:2468] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.000077 26668 raft_consensus.cc:385] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.003326 26668 raft_consensus.cc:740] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.003165 26588 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
W20260709 12:20:31.004489 26588 leader_election.cc:343] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.004927 26588 leader_election.cc:304] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:31.006825 26721 raft_consensus.cc:2749] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.010316 26721 consensus_queue.cc:1048] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.010375 26438 leader_election.cc:304] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.011930 26734 consensus_queue.cc:1048] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.004163 26668 consensus_queue.cc:260] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.013362 26786 raft_consensus.cc:2804] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.013823 26786 raft_consensus.cc:697] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:31.014667 26786 consensus_queue.cc:237] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.004104 26680 raft_consensus.cc:2749] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:30.986805 26670 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.021162 26670 ts_tablet_manager.cc:1403] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.176s	user 0.012s	sys 0.007s
I20260709 12:20:31.024389 26778 consensus_queue.cc:1048] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.026954 26669 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.027643 26669 ts_tablet_manager.cc:1403] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.286s	user 0.015s	sys 0.002s
I20260709 12:20:31.030542 26669 raft_consensus.cc:359] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.026938 26668 ts_tablet_manager.cc:1434] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:20:31.031337 26669 raft_consensus.cc:385] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.031666 26669 raft_consensus.cc:740] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.032047 26668 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.032480 26669 consensus_queue.cc:260] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.035313 26669 ts_tablet_manager.cc:1434] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:31.036278 26669 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.054338 26668 tablet_bootstrap.cc:654] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.054677 26670 raft_consensus.cc:359] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.055490 26670 raft_consensus.cc:385] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.055788 26670 raft_consensus.cc:740] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.058738 26787 consensus_queue.cc:1048] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.058146 26294 catalog_manager.cc:5697] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.063499 26668 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.063961 26668 ts_tablet_manager.cc:1403] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 12:20:31.066608 26668 raft_consensus.cc:359] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.067261 26668 raft_consensus.cc:385] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.067524 26669 tablet_bootstrap.cc:654] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.067533 26668 raft_consensus.cc:740] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.068548 26668 consensus_queue.cc:260] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.070653 26668 ts_tablet_manager.cc:1434] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:31.071503 26668 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.072907 26298 catalog_manager.cc:5697] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.076210 26722 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.089218 26722 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.092031 26722 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.092233 26787 raft_consensus.cc:493] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.092396 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.092813 26787 raft_consensus.cc:515] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.078814 26750 consensus_queue.cc:1048] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.080055 26670 consensus_queue.cc:260] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.093091 26546 raft_consensus.cc:2468] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:31.093640 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.094861 26366 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
W20260709 12:20:31.095577 26364 leader_election.cc:343] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.095669 26782 raft_consensus.cc:2804] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.096221 26782 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.096643 26782 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.077317 26668 tablet_bootstrap.cc:654] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.089249 26669 tablet_bootstrap.cc:492] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.101050 26669 ts_tablet_manager.cc:1403] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.005s
I20260709 12:20:31.103921 26669 raft_consensus.cc:359] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.104737 26669 raft_consensus.cc:385] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.104923 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.105082 26669 raft_consensus.cc:740] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.105509 26787 leader_election.cc:290] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.105890 26670 ts_tablet_manager.cc:1434] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.002s
I20260709 12:20:31.106861 26670 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.106514 26669 consensus_queue.cc:260] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.109894 26546 raft_consensus.cc:2468] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.111613 26441 leader_election.cc:304] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.112418 26787 raft_consensus.cc:2804] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.112764 26787 raft_consensus.cc:493] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.113335 26787 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.112653 26782 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.116532 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:31.117355 26548 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.120882 26787 raft_consensus.cc:515] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.124370 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:31.124966 26546 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.128599 26670 tablet_bootstrap.cc:654] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.130466 26787 leader_election.cc:290] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.130507 26548 raft_consensus.cc:2468] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:31.142010 26667 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.143249 26667 ts_tablet_manager.cc:1403] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.171s	user 0.020s	sys 0.024s
I20260709 12:20:31.142277 26366 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.144608 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:31.141793 26782 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.145908 26782 raft_consensus.cc:2804] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.146405 26782 raft_consensus.cc:697] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:31.147207 26782 consensus_queue.cc:237] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:31.147601 26364 leader_election.cc:343] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.151094 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:31.151718 26623 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.156206 26788 raft_consensus.cc:493] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.156838 26788 raft_consensus.cc:515] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.171926 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.172108 26667 raft_consensus.cc:359] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.172916 26667 raft_consensus.cc:385] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.174916 26788 leader_election.cc:290] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.175966 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.176383 26788 consensus_queue.cc:1048] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:31.178125 26586 leader_election.cc:343] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.173182 26667 raft_consensus.cc:740] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.191768 26546 raft_consensus.cc:2468] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.183923 26667 consensus_queue.cc:260] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.197610 26441 leader_election.cc:304] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.197876 26667 ts_tablet_manager.cc:1434] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.002s
I20260709 12:20:31.198623 26787 raft_consensus.cc:2804] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.198892 26667 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.199188 26787 raft_consensus.cc:697] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:31.200127 26787 consensus_queue.cc:237] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.201622 26623 raft_consensus.cc:2468] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.202842 26669 ts_tablet_manager.cc:1434] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.001s
I20260709 12:20:31.203640 26668 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.204116 26668 ts_tablet_manager.cc:1403] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.133s	user 0.041s	sys 0.042s
I20260709 12:20:31.213912 26669 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.217519 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:31.214640 26295 catalog_manager.cc:5697] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.221757 26667 tablet_bootstrap.cc:654] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.224920 26668 raft_consensus.cc:359] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.225879 26668 raft_consensus.cc:385] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.226219 26668 raft_consensus.cc:740] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
W20260709 12:20:31.228626 26588 leader_election.cc:343] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.229128 26588 leader_election.cc:304] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:31.229621 26670 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.230331 26788 raft_consensus.cc:2749] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.227118 26668 consensus_queue.cc:260] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.230326 26670 ts_tablet_manager.cc:1403] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.124s	user 0.011s	sys 0.005s
I20260709 12:20:31.232154 26668 ts_tablet_manager.cc:1434] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:20:31.233206 26668 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.233861 26670 raft_consensus.cc:359] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.234571 26670 raft_consensus.cc:385] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.234854 26670 raft_consensus.cc:740] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.235630 26670 consensus_queue.cc:260] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.238088 26670 ts_tablet_manager.cc:1434] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:31.239043 26670 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.241076 26668 tablet_bootstrap.cc:654] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.244918 26699 consensus_queue.cc:1048] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.257907 26667 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.258309 26669 tablet_bootstrap.cc:654] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.258513 26667 ts_tablet_manager.cc:1403] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.014s
I20260709 12:20:31.270498 26667 raft_consensus.cc:359] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.271559 26667 raft_consensus.cc:385] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.271915 26667 raft_consensus.cc:740] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.274420 26670 tablet_bootstrap.cc:654] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.274257 26667 consensus_queue.cc:260] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.275106 26298 catalog_manager.cc:5697] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.282429 26667 ts_tablet_manager.cc:1434] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.005s
I20260709 12:20:31.284114 26667 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.296314 26668 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.296752 26669 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.296885 26668 ts_tablet_manager.cc:1403] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.064s	user 0.009s	sys 0.024s
I20260709 12:20:31.297282 26669 ts_tablet_manager.cc:1403] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.000s
I20260709 12:20:31.298784 26784 consensus_queue.cc:1048] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.300000 26668 raft_consensus.cc:359] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.300715 26668 raft_consensus.cc:385] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.300774 26670 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.301069 26668 raft_consensus.cc:740] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.301471 26670 ts_tablet_manager.cc:1403] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.063s	user 0.009s	sys 0.004s
I20260709 12:20:31.302156 26668 consensus_queue.cc:260] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.303282 26667 tablet_bootstrap.cc:654] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.304574 26668 ts_tablet_manager.cc:1434] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:31.305529 26668 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.305860 26670 raft_consensus.cc:359] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.306753 26670 raft_consensus.cc:385] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.307093 26670 raft_consensus.cc:740] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.311848 26670 consensus_queue.cc:260] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.318610 26670 ts_tablet_manager.cc:1434] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:20:31.319664 26670 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.329493 26668 tablet_bootstrap.cc:654] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.332856 26667 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.333226 26670 tablet_bootstrap.cc:654] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.333530 26667 ts_tablet_manager.cc:1403] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.007s
I20260709 12:20:31.332859 26669 raft_consensus.cc:359] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.335398 26669 raft_consensus.cc:385] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.335752 26669 raft_consensus.cc:740] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.336874 26669 consensus_queue.cc:260] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.348009 26669 ts_tablet_manager.cc:1434] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.004s
I20260709 12:20:31.349007 26669 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.350767 26734 raft_consensus.cc:493] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.351289 26734 raft_consensus.cc:515] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.353616 26734 leader_election.cc:290] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.354750 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.355465 26400 raft_consensus.cc:2468] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.356593 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.357235 26438 leader_election.cc:304] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.359040 26786 raft_consensus.cc:2804] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.359434 26786 raft_consensus.cc:493] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.359751 26786 raft_consensus.cc:3060] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.357287 26622 raft_consensus.cc:2468] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.377866 26667 raft_consensus.cc:359] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.378934 26667 raft_consensus.cc:385] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.379313 26667 raft_consensus.cc:740] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.380180 26667 consensus_queue.cc:260] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.382921 26667 ts_tablet_manager.cc:1434] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 12:20:31.390594 26668 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.391212 26670 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
W20260709 12:20:31.387907 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:31.392249 26668 ts_tablet_manager.cc:1403] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.087s	user 0.021s	sys 0.024s
I20260709 12:20:31.384876 26786 raft_consensus.cc:515] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.394526 26786 leader_election.cc:290] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.387626 26669 tablet_bootstrap.cc:654] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.392309 26670 ts_tablet_manager.cc:1403] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.073s	user 0.020s	sys 0.008s
I20260709 12:20:31.394781 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:31.395421 26622 raft_consensus.cc:3060] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.395133 26668 raft_consensus.cc:359] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.395928 26668 raft_consensus.cc:385] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.396207 26668 raft_consensus.cc:740] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.396874 26668 consensus_queue.cc:260] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.397605 26670 raft_consensus.cc:359] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.398294 26670 raft_consensus.cc:385] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.398573 26670 raft_consensus.cc:740] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.398856 26668 ts_tablet_manager.cc:1434] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:20:31.399230 26670 consensus_queue.cc:260] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.399858 26668 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.401569 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba40df2a50424f46b52f9c4cf6894664" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:31.402230 26400 raft_consensus.cc:3060] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.391600 26667 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.406492 26668 tablet_bootstrap.cc:654] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.406374 26622 raft_consensus.cc:2468] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.408957 26670 ts_tablet_manager.cc:1434] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:20:31.410061 26670 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.410123 26439 leader_election.cc:304] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.411083 26786 raft_consensus.cc:2804] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.411711 26786 raft_consensus.cc:697] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:31.412573 26786 consensus_queue.cc:237] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.416250 26668 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.416783 26668 ts_tablet_manager.cc:1403] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:20:31.418196 26669 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.418680 26669 ts_tablet_manager.cc:1403] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 12:20:31.419685 26668 raft_consensus.cc:359] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.420045 26400 raft_consensus.cc:2468] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.420432 26668 raft_consensus.cc:385] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.420755 26668 raft_consensus.cc:740] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.421510 26668 consensus_queue.cc:260] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.423808 26668 ts_tablet_manager.cc:1434] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:31.423942 26670 tablet_bootstrap.cc:654] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.421548 26669 raft_consensus.cc:359] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.434885 26668 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.434988 26669 raft_consensus.cc:385] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.435415 26669 raft_consensus.cc:740] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.436748 26667 tablet_bootstrap.cc:654] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.439494 26669 consensus_queue.cc:260] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.439811 26734 consensus_queue.cc:1048] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.440846 26668 tablet_bootstrap.cc:654] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.456594 26669 ts_tablet_manager.cc:1434] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:20:31.466303 26734 raft_consensus.cc:493] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.467206 26734 raft_consensus.cc:515] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.470662 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.471385 26400 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.472565 26438 leader_election.cc:304] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.473527 26786 raft_consensus.cc:2804] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.473915 26786 raft_consensus.cc:493] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.474242 26786 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.476109 26669 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.489684 26782 consensus_queue.cc:1048] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.501860 26787 consensus_queue.cc:1048] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:31.502579 26667 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.503084 26667 ts_tablet_manager.cc:1403] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.002s
I20260709 12:20:31.511276 26722 consensus_queue.cc:1048] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.514981 26667 raft_consensus.cc:359] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.515985 26667 raft_consensus.cc:385] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.516364 26667 raft_consensus.cc:740] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.519877 26669 tablet_bootstrap.cc:654] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.523324 26667 consensus_queue.cc:260] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.524966 26670 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.525466 26670 ts_tablet_manager.cc:1403] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.116s	user 0.030s	sys 0.034s
I20260709 12:20:31.530284 26667 ts_tablet_manager.cc:1434] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:20:31.532452 26667 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.538668 26669 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.539219 26669 ts_tablet_manager.cc:1403] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.000s
I20260709 12:20:31.534607 26670 raft_consensus.cc:359] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.541869 26670 raft_consensus.cc:385] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.542148 26670 raft_consensus.cc:740] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.542963 26670 consensus_queue.cc:260] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.544140 26669 raft_consensus.cc:359] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.544845 26669 raft_consensus.cc:385] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.545109 26669 raft_consensus.cc:740] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.545328 26670 ts_tablet_manager.cc:1434] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:20:31.546433 26670 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.547528 26669 consensus_queue.cc:260] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.548642 26786 raft_consensus.cc:515] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.552325 26670 tablet_bootstrap.cc:654] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.556708 26669 ts_tablet_manager.cc:1434] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:20:31.558197 26669 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.563524 26667 tablet_bootstrap.cc:654] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.564620 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.565338 26546 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.566438 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:31.567265 26400 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.571067 26669 tablet_bootstrap.cc:654] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.575296 26734 leader_election.cc:290] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:31.578277 26670 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.578945 26670 ts_tablet_manager.cc:1403] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.000s
I20260709 12:20:31.580171 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:31.580983 26546 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.582643 26670 raft_consensus.cc:359] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.585055 26670 raft_consensus.cc:385] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.585318 26670 raft_consensus.cc:740] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.586086 26667 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.586138 26670 consensus_queue.cc:260] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.586732 26667 ts_tablet_manager.cc:1403] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.055s	user 0.006s	sys 0.006s
I20260709 12:20:31.588795 26670 ts_tablet_manager.cc:1434] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:31.589568 26667 raft_consensus.cc:359] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.590065 26670 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.590288 26667 raft_consensus.cc:385] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.590570 26667 raft_consensus.cc:740] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.591411 26667 consensus_queue.cc:260] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.593189 26667 ts_tablet_manager.cc:1434] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:31.594130 26667 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.597942 26670 tablet_bootstrap.cc:654] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.600188 26667 tablet_bootstrap.cc:654] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.606263 26669 tablet_bootstrap.cc:492] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.606254 26787 consensus_queue.cc:1048] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.607021 26669 ts_tablet_manager.cc:1403] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.006s
I20260709 12:20:31.612547 26670 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.613085 26670 ts_tablet_manager.cc:1403] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:20:31.614877 26669 raft_consensus.cc:359] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.615624 26669 raft_consensus.cc:385] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.622694 26669 raft_consensus.cc:740] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.623797 26669 consensus_queue.cc:260] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.626618 26669 ts_tablet_manager.cc:1434] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:20:31.629123 26787 consensus_queue.cc:1048] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.616173 26670 raft_consensus.cc:359] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.618603 26788 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.635825 26670 raft_consensus.cc:385] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.630892 26669 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.636337 26788 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.643332 26670 raft_consensus.cc:740] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.644470 26670 consensus_queue.cc:260] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.646752 26788 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.644766 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.658591 26548 raft_consensus.cc:2393] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:31.606032 26546 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.660372 26669 tablet_bootstrap.cc:654] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.663093 26784 raft_consensus.cc:493] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.651767 26670 ts_tablet_manager.cc:1434] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 12:20:31.655660 26699 raft_consensus.cc:493] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.664414 26668 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.664582 26699 raft_consensus.cc:515] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.648980 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.661834 26441 leader_election.cc:304] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.663712 26784 raft_consensus.cc:515] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.667841 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84283d0aa91944d482d29b9d70c80299" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.668741 26397 raft_consensus.cc:2468] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:31.669124 26586 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:31.669793 26667 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.670284 26667 ts_tablet_manager.cc:1403] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.076s	user 0.029s	sys 0.012s
I20260709 12:20:31.670202 26512 leader_election.cc:304] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.671420 26731 raft_consensus.cc:2804] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.671703 26765 raft_consensus.cc:2804] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.672035 26731 raft_consensus.cc:697] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:31.672329 26765 raft_consensus.cc:493] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.672843 26765 raft_consensus.cc:3060] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.673069 26731 consensus_queue.cc:237] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.664921 26670 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.667979 26784 leader_election.cc:290] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.676175 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84283d0aa91944d482d29b9d70c80299" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:31.676855 26474 raft_consensus.cc:2468] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:31.678097 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:31.678912 26475 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:31.680876 26514 leader_election.cc:304] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.681725 26670 tablet_bootstrap.cc:654] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.681860 26784 raft_consensus.cc:2804] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.682241 26784 raft_consensus.cc:493] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.682626 26784 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.673530 26667 raft_consensus.cc:359] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.683568 26667 raft_consensus.cc:385] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.683854 26667 raft_consensus.cc:740] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.683983 26765 raft_consensus.cc:515] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.684721 26667 consensus_queue.cc:260] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.687330 26667 ts_tablet_manager.cc:1434] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.002s
I20260709 12:20:31.687737 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84283d0aa91944d482d29b9d70c80299" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:31.668023 26699 leader_election.cc:290] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.688496 26397 raft_consensus.cc:3060] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.688376 26667 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.688099 26789 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.664930 26668 ts_tablet_manager.cc:1403] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.240s	user 0.072s	sys 0.139s
I20260709 12:20:31.554518 26298 catalog_manager.cc:5697] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:31.600118 26786 leader_election.cc:290] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:31.669160 26400 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:31.705083 26667 tablet_bootstrap.cc:654] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.707113 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.711166 26784 raft_consensus.cc:515] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.697407 26788 raft_consensus.cc:493] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.724596 26668 raft_consensus.cc:359] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.726317 26668 raft_consensus.cc:385] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.726687 26668 raft_consensus.cc:740] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.735855 26788 raft_consensus.cc:515] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.745051 26765 leader_election.cc:290] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.761754 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:31.762388 26475 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.763295 26668 consensus_queue.cc:260] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.764886 26668 ts_tablet_manager.cc:1434] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.002s
I20260709 12:20:31.765096 26722 raft_consensus.cc:493] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.765666 26621 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:31.765564 26722 raft_consensus.cc:515] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.769394 26722 leader_election.cc:290] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.771202 26668 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:31.772244 26699 raft_consensus.cc:493] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.772516 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.772769 26699 raft_consensus.cc:515] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.774995 26699 leader_election.cc:290] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.758399 26784 leader_election.cc:290] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.779636 26548 raft_consensus.cc:2468] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:31.784036 26366 leader_election.cc:304] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.785079 26782 consensus_queue.cc:1048] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.785318 26722 raft_consensus.cc:2804] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.786082 26722 raft_consensus.cc:493] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.786614 26722 raft_consensus.cc:3060] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.791312 26847 raft_consensus.cc:493] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.791939 26847 raft_consensus.cc:515] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.796011 26782 raft_consensus.cc:493] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.796492 26782 raft_consensus.cc:515] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.797379 26789 raft_consensus.cc:2749] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.798051 26789 raft_consensus.cc:493] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.798149 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.798657 26782 leader_election.cc:290] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.798506 26789 raft_consensus.cc:515] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.794757 26397 raft_consensus.cc:2468] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:31.805006 26670 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:31.805562 26670 ts_tablet_manager.cc:1403] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.141s	user 0.026s	sys 0.016s
I20260709 12:20:31.804252 26475 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:31.810014 26668 tablet_bootstrap.cc:654] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.809710 26670 raft_consensus.cc:359] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.810425 26670 raft_consensus.cc:385] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.810720 26670 raft_consensus.cc:740] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.813355 26669 tablet_bootstrap.cc:492] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:31.813913 26669 ts_tablet_manager.cc:1403] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.183s	user 0.031s	sys 0.024s
I20260709 12:20:31.816063 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.816190 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.814797 26670 consensus_queue.cc:260] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.816063 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.821025 26621 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:31.817442 26547 raft_consensus.cc:2468] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:31.816321 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.823508 26669 raft_consensus.cc:359] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.824543 26669 raft_consensus.cc:385] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.825002 26669 raft_consensus.cc:740] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.825604 26669 consensus_queue.cc:260] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.819262 26622 raft_consensus.cc:2468] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:31.829598 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84283d0aa91944d482d29b9d70c80299" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:31.830340 26475 raft_consensus.cc:3060] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.830382 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:31.831300 26473 raft_consensus.cc:2393] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate 297900273c7740af9938749b3ce3a80d in this term.
I20260709 12:20:31.831452 26545 raft_consensus.cc:2393] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate 297900273c7740af9938749b3ce3a80d in this term.
I20260709 12:20:31.831907 26589 leader_election.cc:304] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:31.833369 26846 raft_consensus.cc:2804] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.833736 26846 raft_consensus.cc:493] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.834028 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.819965 26512 leader_election.cc:304] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.834048 26846 raft_consensus.cc:3060] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.836380 26669 ts_tablet_manager.cc:1434] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:20:31.837177 26670 ts_tablet_manager.cc:1434] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.003s
I20260709 12:20:31.837651 26669 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:31.839509 26670 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:31.842713 26364 leader_election.cc:304] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:31.843650 26782 raft_consensus.cc:2804] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.844043 26782 raft_consensus.cc:493] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.824549 26514 leader_election.cc:304] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.844398 26782 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.845177 26849 raft_consensus.cc:2804] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.847524 26848 raft_consensus.cc:2804] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:31.848294 26848 raft_consensus.cc:697] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:31.849154 26849 raft_consensus.cc:697] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:31.849464 26848 consensus_queue.cc:237] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:31.850252 26849 consensus_queue.cc:237] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:31.852414 26589 leader_election.cc:343] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.854470 26788 leader_election.cc:290] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:31.854919 26475 raft_consensus.cc:2468] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:31.866567 26667 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:31.869477 26667 ts_tablet_manager.cc:1403] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.181s	user 0.008s	sys 0.007s
I20260709 12:20:31.876637 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:31.877576 26474 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:31.882637 26847 leader_election.cc:290] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.882828 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:31.907788 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.914126 26621 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.923617 26789 leader_election.cc:290] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:31.925190 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.926026 26397 raft_consensus.cc:2468] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:31.926745 26294 catalog_manager.cc:5697] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:31.930361 26586 leader_election.cc:304] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:31.933243 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:31.884220 26588 leader_election.cc:304] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:31.935461 26669 tablet_bootstrap.cc:654] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.935928 26789 raft_consensus.cc:2804] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.936479 26789 raft_consensus.cc:493] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.936915 26789 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:31.940047 26588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.66:37205, user_credentials={real_user=slave}} blocked reactor thread for 55952.4us
I20260709 12:20:31.944770 26778 consensus_queue.cc:1048] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.913733 26667 raft_consensus.cc:359] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.955508 26667 raft_consensus.cc:385] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.955906 26667 raft_consensus.cc:740] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.958469 26667 consensus_queue.cc:260] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.912189 26740 raft_consensus.cc:493] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.961822 26740 raft_consensus.cc:515] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:31.963587 26668 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:31.964087 26668 ts_tablet_manager.cc:1403] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.194s	user 0.013s	sys 0.028s
I20260709 12:20:31.965536 26740 leader_election.cc:290] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:31.912298 26782 raft_consensus.cc:515] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.959013 26786 consensus_queue.cc:1048] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:31.910085 26846 raft_consensus.cc:515] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.971832 26667 ts_tablet_manager.cc:1434] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 12:20:31.974933 26667 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:31.976624 26668 raft_consensus.cc:359] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:31.977223 26668 raft_consensus.cc:385] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:31.977427 26668 raft_consensus.cc:740] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:31.900241 26722 raft_consensus.cc:515] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:31.974607 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:31.987650 26545 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.989097 26441 leader_election.cc:304] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:31.974323 26782 leader_election.cc:290] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.979513 26619 raft_consensus.cc:2393] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:31.990341 26786 raft_consensus.cc:2804] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:31.990775 26786 raft_consensus.cc:493] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.980098 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:31.991168 26786 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.991545 26475 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.995672 26399 raft_consensus.cc:2393] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:31.995949 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:31.996600 26623 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:31.981043 26621 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:31.981297 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:31.998767 26622 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:31.999418 26787 consensus_queue.cc:1048] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.000067 26852 raft_consensus.cc:2749] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.001080 26670 tablet_bootstrap.cc:654] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:31.981331 26778 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.003494 26778 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.002828 26512 leader_election.cc:304] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.005776 26778 leader_election.cc:290] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:31.993139 26789 raft_consensus.cc:515] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.010262 26789 leader_election.cc:290] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.011754 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:32.012390 26399 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.015020 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:32.016374 26849 raft_consensus.cc:3060] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.016881 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
W20260709 12:20:32.018679 26364 leader_election.cc:343] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.989192 26295 catalog_manager.cc:5697] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:32.024921 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.026103 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.025980 26295 catalog_manager.cc:5697] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.027968 26849 raft_consensus.cc:2749] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:31.982677 26846 leader_election.cc:290] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.024282 26852 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.023029 26399 raft_consensus.cc:2468] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:32.028927 26852 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.031689 26852 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
W20260709 12:20:32.027911 26589 leader_election.cc:343] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:31.984163 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:32.032823 26396 raft_consensus.cc:2393] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:32.035565 26722 leader_election.cc:290] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:32.037052 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.018880 26475 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:32.040822 26586 leader_election.cc:304] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:32.041139 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:32.047206 26365 leader_election.cc:304] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:32.048280 26782 raft_consensus.cc:2804] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.048844 26782 raft_consensus.cc:697] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:32.049122 26622 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:32.049646 26782 consensus_queue.cc:237] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.059582 26669 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:32.060124 26669 ts_tablet_manager.cc:1403] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.223s	user 0.018s	sys 0.034s
I20260709 12:20:31.983311 26668 consensus_queue.cc:260] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.069500 26667 tablet_bootstrap.cc:654] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.077325 26668 ts_tablet_manager.cc:1434] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.000s
I20260709 12:20:32.067780 26852 raft_consensus.cc:2804] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.067361 26473 raft_consensus.cc:2468] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:32.067444 26548 raft_consensus.cc:2468] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
W20260709 12:20:32.075699 26589 leader_election.cc:343] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.062961 26669 raft_consensus.cc:359] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.087947 26669 raft_consensus.cc:385] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.088306 26669 raft_consensus.cc:740] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.090876 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:32.093865 26589 leader_election.cc:304] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:32.094775 26789 raft_consensus.cc:2804] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.095405 26789 raft_consensus.cc:697] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:32.066316 26786 raft_consensus.cc:515] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.096437 26789 consensus_queue.cc:237] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.100033 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.100732 26548 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.083740 26365 leader_election.cc:304] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:32.102036 26722 raft_consensus.cc:2804] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:32.102419 26722 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.102759 26722 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.102989 26734 consensus_queue.cc:1048] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.083949 26740 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.102800 26669 consensus_queue.cc:260] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.105860 26740 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:32.107781 26586 leader_election.cc:343] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.108774 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97abcb3c90494ee2b8ad8f45c34923f5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:32.111011 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:32.111790 26740 leader_election.cc:290] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.106313 26786 leader_election.cc:290] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:32.085618 26852 raft_consensus.cc:697] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:32.113262 26852 consensus_queue.cc:237] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.115288 26586 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.116091 26788 raft_consensus.cc:2749] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.124310 26668 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:32.119179 26669 ts_tablet_manager.cc:1434] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 12:20:32.128855 26669 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:32.133147 26364 leader_election.cc:304] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:32.181543 26866 raft_consensus.cc:2749] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.196852 26670 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:32.197436 26670 ts_tablet_manager.cc:1403] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.360s	user 0.009s	sys 0.012s
W20260709 12:20:32.195066 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 92910.2us
W20260709 12:20:32.201822 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 68788.3us
I20260709 12:20:32.203903 26668 tablet_bootstrap.cc:654] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.204663 26722 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.206552 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:32.209632 26670 raft_consensus.cc:359] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.210433 26670 raft_consensus.cc:385] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.210803 26396 raft_consensus.cc:2393] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:32.207178 26621 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.218566 26722 leader_election.cc:290] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:32.219376 26667 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:32.219934 26667 ts_tablet_manager.cc:1403] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.247s	user 0.016s	sys 0.002s
I20260709 12:20:32.221911 26295 catalog_manager.cc:5697] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.210760 26670 raft_consensus.cc:740] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.223135 26295 catalog_manager.cc:5697] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.223855 26670 consensus_queue.cc:260] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.222836 26667 raft_consensus.cc:359] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.227540 26667 raft_consensus.cc:385] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.227849 26667 raft_consensus.cc:740] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.231443 26669 tablet_bootstrap.cc:654] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.230170 26667 consensus_queue.cc:260] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.238086 26667 ts_tablet_manager.cc:1434] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:20:32.239526 26667 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:32.242395 26868 raft_consensus.cc:493] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.236029 26621 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:32.242954 26868 raft_consensus.cc:515] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.245004 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:32.245679 26618 raft_consensus.cc:2468] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:32.245896 26868 leader_election.cc:290] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:32.247664 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:32.252421 26366 leader_election.cc:343] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.254448 26670 ts_tablet_manager.cc:1434] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:20:32.255910 26670 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:32.256502 26548 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:32.264613 26667 tablet_bootstrap.cc:654] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.254159 26439 leader_election.cc:304] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:32.257681 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:32.259923 26298 catalog_manager.cc:5697] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.260746 26668 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:32.271728 26621 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.271982 26668 ts_tablet_manager.cc:1403] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.148s	user 0.015s	sys 0.008s
I20260709 12:20:32.274183 26439 leader_election.cc:304] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.274840 26668 raft_consensus.cc:359] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.275780 26668 raft_consensus.cc:385] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.276036 26668 raft_consensus.cc:740] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.276759 26668 consensus_queue.cc:260] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.277791 26787 raft_consensus.cc:2804] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.280064 26787 raft_consensus.cc:697] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:32.280958 26787 consensus_queue.cc:237] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.282595 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:32.283015 26621 raft_consensus.cc:2468] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:32.283596 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:32.277863 26786 raft_consensus.cc:2804] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:32.284611 26786 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.285030 26786 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.289582 26364 leader_election.cc:304] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:32.290861 26868 raft_consensus.cc:2804] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.291505 26868 raft_consensus.cc:697] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:32.295464 26849 consensus_queue.cc:1048] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.297159 26668 ts_tablet_manager.cc:1434] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
W20260709 12:20:32.298131 26365 leader_election.cc:343] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.298657 26365 leader_election.cc:304] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:32.300477 26722 raft_consensus.cc:2749] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.300539 26786 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.301857 26474 raft_consensus.cc:2393] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:32.305696 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:32.293017 26868 consensus_queue.cc:237] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.314858 26786 leader_election.cc:290] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.321269 26669 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:32.321816 26669 ts_tablet_manager.cc:1403] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.193s	user 0.018s	sys 0.028s
I20260709 12:20:32.322731 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:32.323640 26621 raft_consensus.cc:2393] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:32.324781 26669 raft_consensus.cc:359] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.325520 26669 raft_consensus.cc:385] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.325852 26669 raft_consensus.cc:740] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.323137 26670 tablet_bootstrap.cc:654] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.325640 26668 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:32.326496 26669 consensus_queue.cc:260] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.331673 26298 catalog_manager.cc:5697] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.333693 26669 ts_tablet_manager.cc:1434] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:32.334937 26669 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:32.344308 26439 leader_election.cc:304] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.359124 26667 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:32.361284 26668 tablet_bootstrap.cc:654] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.361914 26849 raft_consensus.cc:493] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.364496 26667 ts_tablet_manager.cc:1403] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.006s
I20260709 12:20:32.362219 26669 tablet_bootstrap.cc:654] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.365206 26731 raft_consensus.cc:2749] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.365255 26848 consensus_queue.cc:1048] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:20:32.370760 26787 consensus_queue.cc:1048] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:32.367961 26849 raft_consensus.cc:515] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.373505 26849 leader_election.cc:290] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.376933 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:32.401861 26669 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:32.405295 26667 raft_consensus.cc:359] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.392184 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:32.416020 26397 raft_consensus.cc:2393] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:32.416908 26667 raft_consensus.cc:385] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.417416 26512 leader_election.cc:304] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.414381 26848 consensus_queue.cc:1048] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:32.418872 26669 ts_tablet_manager.cc:1403] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.007s
I20260709 12:20:32.418955 26667 raft_consensus.cc:740] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.423130 26667 consensus_queue.cc:260] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.425400 26849 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.428072 26667 ts_tablet_manager.cc:1434] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:20:32.429597 26667 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:32.438133 26885 consensus_queue.cc:1048] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:32.439596 26670 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:32.440090 26670 ts_tablet_manager.cc:1403] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.184s	user 0.019s	sys 0.003s
I20260709 12:20:32.466434 26670 raft_consensus.cc:359] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.471308 26849 raft_consensus.cc:2749] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.471922 26670 raft_consensus.cc:385] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.466946 26669 raft_consensus.cc:359] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.472906 26670 raft_consensus.cc:740] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.473270 26669 raft_consensus.cc:385] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.469969 26294 catalog_manager.cc:5697] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.473984 26669 raft_consensus.cc:740] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.479719 26668 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:32.480228 26668 ts_tablet_manager.cc:1403] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.170s	user 0.012s	sys 0.019s
I20260709 12:20:32.482872 26668 raft_consensus.cc:359] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.487600 26668 raft_consensus.cc:385] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.487876 26668 raft_consensus.cc:740] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.488550 26668 consensus_queue.cc:260] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.491524 26668 ts_tablet_manager.cc:1434] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 12:20:32.494443 26669 consensus_queue.cc:260] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.497591 26667 tablet_bootstrap.cc:654] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.498282 26669 ts_tablet_manager.cc:1434] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:20:32.473938 26670 consensus_queue.cc:260] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:32.500481 26513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 79569.9us
I20260709 12:20:32.501894 26670 ts_tablet_manager.cc:1434] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 12:20:32.502897 26670 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:32.492553 26668 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:32.512501 26668 tablet_bootstrap.cc:654] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.519758 26669 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:32.526849 26667 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:32.527639 26667 ts_tablet_manager.cc:1403] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.000s
I20260709 12:20:32.529430 26670 tablet_bootstrap.cc:654] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.527741 26669 tablet_bootstrap.cc:654] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.533361 26667 raft_consensus.cc:359] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.534081 26667 raft_consensus.cc:385] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.538645 26667 raft_consensus.cc:740] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.539368 26667 consensus_queue.cc:260] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.541512 26667 ts_tablet_manager.cc:1434] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:20:32.546612 26667 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:32.559741 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:32.559836 26668 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:32.560261 26787 raft_consensus.cc:493] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.560868 26668 ts_tablet_manager.cc:1403] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.004s
I20260709 12:20:32.560844 26787 raft_consensus.cc:515] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.561880 26667 tablet_bootstrap.cc:654] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.563933 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eeaddd79764e06b3ac77e3c20ffd49" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:32.564636 26668 raft_consensus.cc:359] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.565606 26668 raft_consensus.cc:385] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:32.565716 26438 leader_election.cc:343] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.566004 26668 raft_consensus.cc:740] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.566962 26668 consensus_queue.cc:260] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.570066 26668 ts_tablet_manager.cc:1434] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:20:32.570339 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eeaddd79764e06b3ac77e3c20ffd49" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:32.571281 26668 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:32.571223 26548 raft_consensus.cc:2468] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:32.575785 26787 leader_election.cc:290] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:32.576785 26441 leader_election.cc:304] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.578691 26863 raft_consensus.cc:2804] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:32.579087 26863 raft_consensus.cc:493] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.579452 26863 raft_consensus.cc:3060] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.584621 26852 consensus_queue.cc:1048] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.598165 26863 raft_consensus.cc:515] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.601030 26863 leader_election.cc:290] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:32.602396 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eeaddd79764e06b3ac77e3c20ffd49" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:32.603390 26668 tablet_bootstrap.cc:654] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:32.604177 26438 leader_election.cc:343] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.606168 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6eeaddd79764e06b3ac77e3c20ffd49" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.606791 26548 raft_consensus.cc:3060] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.611794 26852 consensus_queue.cc:1048] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:32.625701 26548 raft_consensus.cc:2468] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:32.627343 26441 leader_election.cc:304] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.634449 26787 raft_consensus.cc:2804] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.634979 26787 raft_consensus.cc:697] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:32.635898 26787 consensus_queue.cc:237] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.644838 26868 consensus_queue.cc:1048] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.645336 26670 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:32.645846 26670 ts_tablet_manager.cc:1403] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.143s	user 0.030s	sys 0.016s
I20260709 12:20:32.650346 26669 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:32.651100 26669 ts_tablet_manager.cc:1403] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.132s	user 0.006s	sys 0.041s
I20260709 12:20:32.654654 26668 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:32.655376 26668 ts_tablet_manager.cc:1403] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.084s	user 0.014s	sys 0.000s
I20260709 12:20:32.657732 26667 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:32.658360 26667 ts_tablet_manager.cc:1403] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.112s	user 0.035s	sys 0.026s
I20260709 12:20:32.659085 26670 raft_consensus.cc:359] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.659793 26670 raft_consensus.cc:385] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.660073 26670 raft_consensus.cc:740] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.661860 26667 raft_consensus.cc:359] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.662358 26868 consensus_queue.cc:1048] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.662735 26667 raft_consensus.cc:385] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.663192 26667 raft_consensus.cc:740] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.664219 26667 consensus_queue.cc:260] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.667371 26667 ts_tablet_manager.cc:1434] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:20:32.668174 26786 consensus_queue.cc:1048] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.668577 26667 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:32.678467 26788 consensus_queue.cc:1048] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.680341 26669 raft_consensus.cc:359] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.681071 26669 raft_consensus.cc:385] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.681363 26669 raft_consensus.cc:740] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.686401 26670 consensus_queue.cc:260] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.683072 26669 consensus_queue.cc:260] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.696089 26669 ts_tablet_manager.cc:1434] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 12:20:32.698024 26670 ts_tablet_manager.cc:1434] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.002s
I20260709 12:20:32.698378 26868 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.700938 26670 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:32.701658 26669 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:32.711716 26668 raft_consensus.cc:359] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.712333 26668 raft_consensus.cc:385] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.712765 26668 raft_consensus.cc:740] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.714289 26668 consensus_queue.cc:260] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.702332 26868 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.720625 26668 ts_tablet_manager.cc:1434] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 12:20:32.722407 26668 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:32.731971 26868 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:32.733316 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:32.734031 26622 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:32.738960 26789 consensus_queue.cc:1048] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:32.739176 26364 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:32.740159 26722 raft_consensus.cc:2804] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:32.740665 26722 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.741055 26722 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.743347 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:32.744153 26546 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:32.754107 26669 tablet_bootstrap.cc:654] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.755463 26667 tablet_bootstrap.cc:654] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.753677 26722 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.759426 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.760077 26546 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.771725 26722 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:32.774821 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:32.775600 26622 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.776775 26670 tablet_bootstrap.cc:654] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.789577 26863 consensus_queue.cc:1048] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:32.792351 26885 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.793057 26885 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.796502 26885 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.798182 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:32.798513 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:32.799882 26513 leader_election.cc:343] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:32.801360 26512 leader_election.cc:343] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.801955 26512 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:32.803364 26885 raft_consensus.cc:2749] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:32.804311 26546 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:32.804616 26298 catalog_manager.cc:5697] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:32.805495 26366 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:32.807562 26669 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:32.808184 26669 ts_tablet_manager.cc:1403] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.009s
I20260709 12:20:32.812824 26669 raft_consensus.cc:359] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.813685 26669 raft_consensus.cc:385] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.814006 26669 raft_consensus.cc:740] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.814807 26669 consensus_queue.cc:260] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.821604 26722 raft_consensus.cc:2804] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:32.865172 26863 raft_consensus.cc:493] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.865576 26722 raft_consensus.cc:697] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:32.865940 26668 tablet_bootstrap.cc:654] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.865696 26863 raft_consensus.cc:515] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.866472 26722 consensus_queue.cc:237] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.869482 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:32.818210 26670 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
W20260709 12:20:32.871660 26441 leader_election.cc:343] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.840651 26667 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:32.834288 26622 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:32.821710 26669 ts_tablet_manager.cc:1434] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:20:32.872993 26667 ts_tablet_manager.cc:1403] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.205s	user 0.030s	sys 0.031s
I20260709 12:20:32.873863 26669 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:32.874454 26911 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.875010 26911 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:32.875937 26667 raft_consensus.cc:359] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.876607 26667 raft_consensus.cc:385] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.876904 26667 raft_consensus.cc:740] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.877374 26911 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:32.877534 26667 consensus_queue.cc:260] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:32.872589 26670 ts_tablet_manager.cc:1403] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.172s	user 0.028s	sys 0.029s
I20260709 12:20:32.869688 26863 leader_election.cc:290] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:32.840967 26789 raft_consensus.cc:493] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.880101 26789 raft_consensus.cc:515] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.882386 26789 leader_election.cc:290] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:32.884130 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:32.885076 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:32.885617 26589 leader_election.cc:343] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:32.885704 26396 raft_consensus.cc:2468] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:32.887151 26586 leader_election.cc:304] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:32.888319 26789 raft_consensus.cc:2804] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:32.888690 26789 raft_consensus.cc:493] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.889050 26789 raft_consensus.cc:3060] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.897433 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:32.899354 26667 ts_tablet_manager.cc:1434] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:20:32.898303 26475 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:32.904996 26667 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:32.900457 26911 raft_consensus.cc:493] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.915443 26911 raft_consensus.cc:515] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.925522 26669 tablet_bootstrap.cc:654] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:32.927418 26670 raft_consensus.cc:359] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.928340 26670 raft_consensus.cc:385] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:32.928671 26670 raft_consensus.cc:740] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:32.929672 26670 consensus_queue.cc:260] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:32.932331 26670 ts_tablet_manager.cc:1434] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 12:20:32.933579 26670 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:32.932232 26789 raft_consensus.cc:515] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:32.934785 26668 tablet_bootstrap.cc:492] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:32.936713 26668 ts_tablet_manager.cc:1403] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.215s	user 0.012s	sys 0.022s
I20260709 12:20:32.938445 26789 leader_election.cc:290] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
W20260709 12:20:32.900512 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 111873us
I20260709 12:20:32.981343 26365 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:32.984728 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:32.986860 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:32.987867 26396 raft_consensus.cc:3060] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.993737 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:32.995218 26667 tablet_bootstrap.cc:654] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.009115 26788 raft_consensus.cc:493] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.007311 26722 raft_consensus.cc:2804] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.022328 26722 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.023041 26722 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:32.999519 26731 raft_consensus.cc:493] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.024740 26788 raft_consensus.cc:515] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:32.996861 26589 leader_election.cc:343] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.025050 26731 raft_consensus.cc:515] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.016577 26670 tablet_bootstrap.cc:654] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.040792 26848 raft_consensus.cc:493] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.037596 26669 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:33.041620 26848 raft_consensus.cc:515] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.042701 26669 ts_tablet_manager.cc:1403] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.001s
I20260709 12:20:33.046404 26848 leader_election.cc:290] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.050513 26788 leader_election.cc:290] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.032457 26668 raft_consensus.cc:359] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.051411 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.051956 26668 raft_consensus.cc:385] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.052281 26547 raft_consensus.cc:2468] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:33.054139 26589 leader_election.cc:304] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:33.056766 26731 leader_election.cc:290] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.057466 26789 raft_consensus.cc:2804] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.057901 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7345dd68527a40889384fe8d3710abda" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:33.060109 26441 leader_election.cc:343] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.062465 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:33.052425 26668 raft_consensus.cc:740] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.058000 26789 raft_consensus.cc:493] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.067428 26789 raft_consensus.cc:3060] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.070091 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.063273 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:33.074687 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:33.075151 26910 consensus_queue.cc:1048] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:33.075678 26911 leader_election.cc:290] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.076757 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7345dd68527a40889384fe8d3710abda" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.077673 26621 raft_consensus.cc:2468] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:33.080282 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.081851 26439 leader_election.cc:304] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d
W20260709 12:20:33.082072 26366 leader_election.cc:343] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.072196 26398 raft_consensus.cc:2468] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:33.089393 26668 consensus_queue.cc:260] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.090750 26473 raft_consensus.cc:2468] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:33.092696 26786 raft_consensus.cc:2804] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.093209 26786 raft_consensus.cc:493] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.093637 26786 raft_consensus.cc:3060] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.092451 26731 raft_consensus.cc:493] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.108616 26731 raft_consensus.cc:515] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.111917 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.112627 26294 catalog_manager.cc:5697] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:33.113490 26364 leader_election.cc:343] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.115123 26911 consensus_queue.cc:1048] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:33.112675 26546 raft_consensus.cc:2468] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:33.116979 26441 leader_election.cc:304] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:33.118088 26915 raft_consensus.cc:2804] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.118497 26915 raft_consensus.cc:493] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.118880 26915 raft_consensus.cc:3060] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.121985 26668 ts_tablet_manager.cc:1434] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.183s	user 0.002s	sys 0.010s
I20260709 12:20:33.124104 26667 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:33.124846 26667 ts_tablet_manager.cc:1403] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.225s	user 0.022s	sys 0.012s
I20260709 12:20:33.125057 26670 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:33.125581 26396 raft_consensus.cc:2468] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:33.125910 26670 ts_tablet_manager.cc:1403] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.193s	user 0.017s	sys 0.019s
I20260709 12:20:33.127403 26586 leader_election.cc:304] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:33.127982 26669 raft_consensus.cc:359] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.128834 26669 raft_consensus.cc:385] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.129187 26669 raft_consensus.cc:740] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.130100 26669 consensus_queue.cc:260] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.129402 26789 raft_consensus.cc:515] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.131717 26622 raft_consensus.cc:2393] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:33.134372 26513 leader_election.cc:304] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:33.135594 26788 raft_consensus.cc:2804] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.136826 26788 raft_consensus.cc:697] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:33.138126 26788 consensus_queue.cc:237] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.138514 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:33.139936 26668 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:33.142228 26789 leader_election.cc:290] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.146173 26786 raft_consensus.cc:515] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.172626 26438 leader_election.cc:304] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:33.174649 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7345dd68527a40889384fe8d3710abda" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:33.177476 26441 leader_election.cc:343] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.151001 26848 raft_consensus.cc:3060] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.170169 26667 raft_consensus.cc:359] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.180878 26667 raft_consensus.cc:385] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.181290 26667 raft_consensus.cc:740] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.182099 26667 consensus_queue.cc:260] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.174930 26787 raft_consensus.cc:2804] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.183488 26787 raft_consensus.cc:493] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.183892 26787 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.184507 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:33.186524 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.145689 26669 ts_tablet_manager.cc:1434] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 12:20:33.187476 26622 raft_consensus.cc:2393] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:33.183507 26863 consensus_queue.cc:1048] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:20:33.188728 26669 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:33.169309 26915 raft_consensus.cc:515] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.169860 26731 leader_election.cc:290] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.195408 26475 raft_consensus.cc:2393] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:33.197175 26667 ts_tablet_manager.cc:1434] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.072s	user 0.004s	sys 0.005s
I20260709 12:20:33.202688 26915 leader_election.cc:290] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.204082 26667 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:33.204347 26668 tablet_bootstrap.cc:654] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.204435 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7345dd68527a40889384fe8d3710abda" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:33.205215 26622 raft_consensus.cc:3060] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.211629 26848 raft_consensus.cc:2749] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.201689 26786 leader_election.cc:290] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.202026 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:33.212466 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "269623ffea96490eab12418e6cbd2c01" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:33.213330 26621 raft_consensus.cc:2393] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:33.214938 26439 leader_election.cc:304] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:33.216320 26669 tablet_bootstrap.cc:654] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.218101 26667 tablet_bootstrap.cc:654] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.219183 26915 raft_consensus.cc:2749] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.183804 26670 raft_consensus.cc:359] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.221125 26670 raft_consensus.cc:385] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.221489 26670 raft_consensus.cc:740] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.222812 26670 consensus_queue.cc:260] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.211830 26787 raft_consensus.cc:515] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.226786 26622 raft_consensus.cc:2468] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:33.226406 26722 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.227334 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:33.234994 26586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 71837.9us
I20260709 12:20:33.236092 26670 ts_tablet_manager.cc:1434] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.000s
I20260709 12:20:33.243144 26668 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:33.268472 26668 ts_tablet_manager.cc:1403] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.129s	user 0.012s	sys 0.002s
I20260709 12:20:33.247807 26787 leader_election.cc:290] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.260757 26669 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:33.269222 26669 ts_tablet_manager.cc:1403] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.008s
I20260709 12:20:33.271644 26548 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.284005 26668 raft_consensus.cc:359] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.285121 26668 raft_consensus.cc:385] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.285534 26668 raft_consensus.cc:740] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.286334 26722 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:33.288185 26670 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:33.288683 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.295535 26548 raft_consensus.cc:2468] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:33.297137 26441 leader_election.cc:304] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:33.298071 26787 raft_consensus.cc:2804] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.298609 26787 raft_consensus.cc:697] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:33.300052 26787 consensus_queue.cc:237] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.300412 26547 raft_consensus.cc:2468] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:33.305289 26364 leader_election.cc:304] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:33.306586 26911 raft_consensus.cc:2749] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.308231 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:33.309095 26618 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.318361 26439 leader_election.cc:304] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:33.319672 26915 raft_consensus.cc:2804] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.320538 26915 raft_consensus.cc:697] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:33.321825 26915 consensus_queue.cc:237] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.286526 26668 consensus_queue.cc:260] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.326815 26668 ts_tablet_manager.cc:1434] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 12:20:33.338972 26669 raft_consensus.cc:359] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.352188 26669 raft_consensus.cc:385] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.352818 26669 raft_consensus.cc:740] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.360109 26589 leader_election.cc:304] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:33.362113 26788 raft_consensus.cc:2804] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.362839 26668 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:33.366243 26788 raft_consensus.cc:697] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:33.367189 26910 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.367700 26788 consensus_queue.cc:237] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.367969 26910 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.385630 26298 catalog_manager.cc:5697] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:33.371011 26910 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:33.411933 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:33.412959 26472 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:33.374570 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 69426.2us
I20260709 12:20:33.422585 26669 consensus_queue.cc:260] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.434579 26667 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:33.434973 26669 ts_tablet_manager.cc:1434] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.165s	user 0.009s	sys 0.000s
I20260709 12:20:33.435144 26667 ts_tablet_manager.cc:1403] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.231s	user 0.028s	sys 0.071s
I20260709 12:20:33.436059 26669 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:33.436920 26618 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:33.437956 26667 raft_consensus.cc:359] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.438683 26667 raft_consensus.cc:385] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.438937 26667 raft_consensus.cc:740] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.439924 26667 consensus_queue.cc:260] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:33.422245 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 249906us
W20260709 12:20:33.446060 26438 consensus_peers.cc:597] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:33.446521 26667 ts_tablet_manager.cc:1434] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:20:33.457836 26910 raft_consensus.cc:493] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.458489 26910 raft_consensus.cc:515] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.458262 26670 tablet_bootstrap.cc:654] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.464089 26667 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:33.468379 26669 tablet_bootstrap.cc:654] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.460683 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:33.474942 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.472065 26475 raft_consensus.cc:2468] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:33.484320 26910 leader_election.cc:290] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.503059 26667 tablet_bootstrap.cc:654] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.501253 26294 catalog_manager.cc:5697] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.504472 26870 consensus_queue.cc:1048] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.486003 26472 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:33.509342 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.516302 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:33.526510 26668 tablet_bootstrap.cc:654] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.496567 26364 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:33.526629 26397 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.532816 26722 raft_consensus.cc:2804] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.533834 26868 consensus_queue.cc:1048] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:33.533862 26366 leader_election.cc:343] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.569831 26669 tablet_bootstrap.cc:492] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:33.570550 26669 ts_tablet_manager.cc:1403] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.007s
I20260709 12:20:33.572424 26667 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:33.572836 26667 ts_tablet_manager.cc:1403] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.017s
I20260709 12:20:33.568535 26848 raft_consensus.cc:493] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.571655 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:33.565670 26365 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:33.583909 26669 raft_consensus.cc:359] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.584656 26669 raft_consensus.cc:385] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:33.584980 26589 consensus_peers.cc:597] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:33.585862 26848 raft_consensus.cc:515] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.587970 26848 leader_election.cc:290] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.589448 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.590235 26852 consensus_queue.cc:1048] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:33.571640 26722 raft_consensus.cc:697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:33.593581 26722 consensus_queue.cc:237] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.588146 26397 raft_consensus.cc:2468] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
W20260709 12:20:33.595506 26513 leader_election.cc:343] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.584961 26669 raft_consensus.cc:740] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.596976 26667 raft_consensus.cc:359] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.597189 26669 consensus_queue.cc:260] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:33.597641 26667 raft_consensus.cc:385] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.597895 26667 raft_consensus.cc:740] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.599671 26669 ts_tablet_manager.cc:1434] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:20:33.601534 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:33.602093 26365 leader_election.cc:343] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.602738 26365 leader_election.cc:304] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:33.605590 26911 raft_consensus.cc:2749] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.606128 26669 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:33.585150 26910 raft_consensus.cc:2804] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.607002 26910 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.607478 26910 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.563776 26294 catalog_manager.cc:5697] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:33.617865 26667 consensus_queue.cc:260] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:33.619488 26514 leader_election.cc:343] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.623200 26514 leader_election.cc:304] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
W20260709 12:20:33.612993 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 79716.5us
I20260709 12:20:33.626124 26699 raft_consensus.cc:2749] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:33.626955 26364 leader_election.cc:343] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.618927 26294 catalog_manager.cc:5697] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:33.634790 26670 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:33.635414 26670 ts_tablet_manager.cc:1403] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.348s	user 0.018s	sys 0.000s
I20260709 12:20:33.636152 26934 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.636684 26934 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.638398 26670 raft_consensus.cc:359] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.638779 26934 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.641139 26668 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:33.641636 26668 ts_tablet_manager.cc:1403] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.282s	user 0.018s	sys 0.013s
I20260709 12:20:33.641736 26910 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.643687 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.643930 26667 ts_tablet_manager.cc:1434] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.001s
I20260709 12:20:33.644502 26546 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:33.645082 26667 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:33.639091 26670 raft_consensus.cc:385] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.645606 26670 raft_consensus.cc:740] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.644498 26668 raft_consensus.cc:359] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.646775 26668 raft_consensus.cc:385] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.647089 26668 raft_consensus.cc:740] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.654102 26669 tablet_bootstrap.cc:654] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.655589 26366 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:33.656536 26911 raft_consensus.cc:2804] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.656939 26911 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.657361 26911 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.659904 26668 consensus_queue.cc:260] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.661305 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:33.662030 26472 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.667723 26668 ts_tablet_manager.cc:1434] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:20:33.668154 26910 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:33.668931 26668 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:33.675410 26667 tablet_bootstrap.cc:654] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.681120 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.690766 26668 tablet_bootstrap.cc:654] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.680120 26670 consensus_queue.cc:260] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.697085 26618 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:33.704917 26911 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.708264 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:33.708822 26670 ts_tablet_manager.cc:1434] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 12:20:33.708940 26546 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.710124 26670 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:33.716143 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:33.726011 26911 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:33.728112 26398 raft_consensus.cc:3060] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.734696 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:33.735404 26621 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.750985 26547 raft_consensus.cc:3060] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.762843 26472 raft_consensus.cc:2468] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
W20260709 12:20:33.764130 26364 leader_election.cc:343] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.766780 26365 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:33.785382 26669 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:33.791885 26722 raft_consensus.cc:2804] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.792568 26669 ts_tablet_manager.cc:1403] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.187s	user 0.024s	sys 0.002s
I20260709 12:20:33.797562 26621 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:33.805001 26722 raft_consensus.cc:697] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:33.805757 26722 consensus_queue.cc:237] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.807688 26670 tablet_bootstrap.cc:654] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.809824 26364 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:33.810949 26911 raft_consensus.cc:2804] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:33.811476 26911 raft_consensus.cc:697] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:33.812269 26787 consensus_queue.cc:1048] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.812778 26668 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:33.812317 26911 consensus_queue.cc:237] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.813325 26668 ts_tablet_manager.cc:1403] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.145s	user 0.037s	sys 0.037s
I20260709 12:20:33.787606 26667 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:33.815649 26667 ts_tablet_manager.cc:1403] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.171s	user 0.013s	sys 0.015s
I20260709 12:20:33.816489 26668 raft_consensus.cc:359] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.817235 26668 raft_consensus.cc:385] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.817510 26668 raft_consensus.cc:740] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.818485 26789 consensus_queue.cc:1048] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.819900 26546 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:33.824843 26787 consensus_queue.cc:1048] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.824717 26668 consensus_queue.cc:260] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.826344 26295 catalog_manager.cc:5697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:33.818898 26667 raft_consensus.cc:359] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.833011 26667 raft_consensus.cc:385] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.833386 26667 raft_consensus.cc:740] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.832249 26669 raft_consensus.cc:359] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.834533 26669 raft_consensus.cc:385] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.834868 26669 raft_consensus.cc:740] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.837878 26668 ts_tablet_manager.cc:1434] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.003s
I20260709 12:20:33.838467 26669 consensus_queue.cc:260] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.839494 26667 consensus_queue.cc:260] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.842664 26667 ts_tablet_manager.cc:1434] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
W20260709 12:20:33.842729 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:33.843719 26667 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:33.845329 26668 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:33.855847 26668 tablet_bootstrap.cc:654] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.870265 26667 tablet_bootstrap.cc:654] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.883713 26919 consensus_queue.cc:1048] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:20:33.887465 26915 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.888008 26915 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.890201 26915 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.890852 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:33.892791 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:33.893260 26438 leader_election.cc:343] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:33.894630 26441 leader_election.cc:343] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.895088 26441 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:33.897413 26731 raft_consensus.cc:2749] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.899276 26787 consensus_queue.cc:1048] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.929553 26670 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:33.929616 26669 ts_tablet_manager.cc:1434] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.000s
I20260709 12:20:33.930238 26670 ts_tablet_manager.cc:1403] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.220s	user 0.018s	sys 0.004s
I20260709 12:20:33.931263 26669 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:33.934618 26668 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:33.935093 26668 ts_tablet_manager.cc:1403] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.019s
I20260709 12:20:33.936546 26667 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:33.937704 26667 ts_tablet_manager.cc:1403] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.003s
I20260709 12:20:33.937753 26668 raft_consensus.cc:359] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.933854 26296 catalog_manager.cc:5697] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:33.938553 26668 raft_consensus.cc:385] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.938861 26668 raft_consensus.cc:740] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.939393 26669 tablet_bootstrap.cc:654] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.942005 26667 raft_consensus.cc:359] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.946062 26667 raft_consensus.cc:385] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.946353 26667 raft_consensus.cc:740] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.944442 26787 consensus_queue.cc:1048] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:33.940002 26670 raft_consensus.cc:359] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.947021 26667 consensus_queue.cc:260] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.947700 26670 raft_consensus.cc:385] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.948125 26670 raft_consensus.cc:740] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.949184 26670 consensus_queue.cc:260] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.949688 26667 ts_tablet_manager.cc:1434] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:20:33.950850 26667 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:33.951087 26786 raft_consensus.cc:493] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.951606 26786 raft_consensus.cc:515] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:33.951979 26670 ts_tablet_manager.cc:1434] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:20:33.953217 26670 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:33.953728 26786 leader_election.cc:290] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:33.956183 26699 raft_consensus.cc:493] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.956985 26667 tablet_bootstrap.cc:654] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.956851 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:33.939599 26296 catalog_manager.cc:5697] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:33.958356 26441 leader_election.cc:343] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:33.959918 26670 tablet_bootstrap.cc:654] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.960388 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:33.961066 26398 raft_consensus.cc:2468] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:33.962174 26438 leader_election.cc:304] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:33.958364 26669 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:33.963248 26787 raft_consensus.cc:2804] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:33.963940 26787 raft_consensus.cc:493] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.964402 26787 raft_consensus.cc:3060] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.945264 26668 consensus_queue.cc:260] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:33.968065 26668 ts_tablet_manager.cc:1434] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:20:33.969208 26668 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:33.956735 26699 raft_consensus.cc:515] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.971513 26699 leader_election.cc:290] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:33.972007 26669 ts_tablet_manager.cc:1403] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.011s
I20260709 12:20:33.973088 26670 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:33.973238 26620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:33.973925 26670 ts_tablet_manager.cc:1403] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.005s
I20260709 12:20:33.975445 26668 tablet_bootstrap.cc:654] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.976738 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d403c93084635b0e4c70ec3829e1c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:33.977772 26670 raft_consensus.cc:359] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.978688 26670 raft_consensus.cc:385] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:33.979091 26670 raft_consensus.cc:740] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:33.978899 26512 leader_election.cc:304] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:33.980396 26699 raft_consensus.cc:2749] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:33.981489 26670 consensus_queue.cc:260] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:33.983739 26670 ts_tablet_manager.cc:1434] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:20:33.984802 26670 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:33.990782 26670 tablet_bootstrap.cc:654] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:33.999998 26667 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.000497 26667 ts_tablet_manager.cc:1403] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.000s
I20260709 12:20:34.003211 26667 raft_consensus.cc:359] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.004269 26667 raft_consensus.cc:385] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.004539 26667 raft_consensus.cc:740] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.004745 26911 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.005283 26911 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.005488 26667 consensus_queue.cc:260] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.008195 26911 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:34.009127 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:34.009652 26667 ts_tablet_manager.cc:1434] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:20:34.010816 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.011593 26667 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:34.012064 26547 raft_consensus.cc:3060] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.009804 26473 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:34.016439 26365 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.017669 26911 raft_consensus.cc:2804] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.018231 26667 tablet_bootstrap.cc:654] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.017791 26787 raft_consensus.cc:515] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.018100 26911 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.018705 26911 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:33.974884 26669 raft_consensus.cc:359] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.020022 26669 raft_consensus.cc:385] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.020326 26669 raft_consensus.cc:740] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.021039 26669 consensus_queue.cc:260] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.021898 26548 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:34.027222 26911 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.033787 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:34.034539 26396 raft_consensus.cc:3060] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.037415 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:34.038033 26473 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.039760 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:34.040401 26548 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.045796 26911 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:34.050485 26548 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:34.051681 26366 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.053949 26722 raft_consensus.cc:2804] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.054436 26722 raft_consensus.cc:697] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:34.055260 26722 consensus_queue.cc:237] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.023104 26669 ts_tablet_manager.cc:1434] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.000s
I20260709 12:20:34.059242 26669 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:34.072027 26396 raft_consensus.cc:2468] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:34.073415 26438 leader_election.cc:304] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.074301 26915 raft_consensus.cc:2804] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.074781 26915 raft_consensus.cc:697] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:34.075824 26787 leader_election.cc:290] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:34.075556 26915 consensus_queue.cc:237] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.076465 26911 consensus_queue.cc:1048] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.079170 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:34.080540 26441 leader_election.cc:343] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.085098 26669 tablet_bootstrap.cc:654] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.094018 26668 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:34.094684 26668 ts_tablet_manager.cc:1403] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.126s	user 0.006s	sys 0.012s
I20260709 12:20:34.100056 26473 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:34.121411 26668 raft_consensus.cc:359] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.122169 26668 raft_consensus.cc:385] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.122474 26668 raft_consensus.cc:740] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.125653 26669 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:34.126329 26669 ts_tablet_manager.cc:1403] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.067s	user 0.007s	sys 0.003s
I20260709 12:20:34.121484 26294 catalog_manager.cc:5697] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:34.123119 26668 consensus_queue.cc:260] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.130873 26668 ts_tablet_manager.cc:1434] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:20:34.132416 26668 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:34.134125 26911 consensus_queue.cc:1048] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.133939 26298 catalog_manager.cc:5697] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:34.135563 26919 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.136119 26919 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.140892 26668 tablet_bootstrap.cc:654] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.142328 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.138633 26919 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:34.145201 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:34.145988 26475 raft_consensus.cc:2393] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:34.149278 26588 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:34.150701 26789 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.162115 26911 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.162688 26911 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.166332 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.167019 26621 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:34.168478 26915 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.168987 26915 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.172195 26915 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:34.174328 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.177366 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.180011 26364 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.181267 26910 raft_consensus.cc:2804] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.181644 26910 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.181986 26910 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.188146 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.188925 26396 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
W20260709 12:20:34.190591 26366 leader_election.cc:343] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.164891 26911 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
W20260709 12:20:34.195653 26439 leader_election.cc:343] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.203079 26438 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:34.203895 26731 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.204295 26731 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.204627 26731 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.209071 26971 consensus_queue.cc:1048] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:34.210330 26910 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.213672 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:34.214210 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:34.214888 26618 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:34.215768 26366 leader_election.cc:343] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.227952 26971 consensus_queue.cc:1048] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.241818 26618 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:34.243189 26364 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:34.244282 26971 raft_consensus.cc:2804] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.245122 26971 raft_consensus.cc:697] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:34.244930 26731 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.246008 26971 consensus_queue.cc:237] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.248034 26789 raft_consensus.cc:2749] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:34.250819 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:34.251209 26910 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
W20260709 12:20:34.252526 26439 leader_election.cc:343] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.255371 26971 consensus_queue.cc:1048] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.276108 26669 raft_consensus.cc:359] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.277102 26669 raft_consensus.cc:385] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.277467 26669 raft_consensus.cc:740] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.282622 26722 consensus_queue.cc:1048] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.278378 26669 consensus_queue.cc:260] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.289841 26669 ts_tablet_manager.cc:1434] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.163s	user 0.010s	sys 0.001s
I20260709 12:20:34.295997 26669 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:34.298063 26731 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:34.303520 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:34.304162 26396 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.313556 26396 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:34.314728 26669 tablet_bootstrap.cc:654] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.316509 26667 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.323519 26667 ts_tablet_manager.cc:1403] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.312s	user 0.014s	sys 0.008s
I20260709 12:20:34.326486 26667 raft_consensus.cc:359] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.327214 26667 raft_consensus.cc:385] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.327569 26667 raft_consensus.cc:740] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.321565 26668 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:34.296193 26294 catalog_manager.cc:5697] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.328424 26438 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:34.329967 26787 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.330799 26787 raft_consensus.cc:697] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:34.329349 26667 consensus_queue.cc:260] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.332043 26787 consensus_queue.cc:237] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.333673 26667 ts_tablet_manager.cc:1434] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:20:34.328512 26668 ts_tablet_manager.cc:1403] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.196s	user 0.066s	sys 0.044s
I20260709 12:20:34.338377 26668 raft_consensus.cc:359] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.339000 26668 raft_consensus.cc:385] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.339314 26668 raft_consensus.cc:740] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.340485 26667 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:34.348258 26699 raft_consensus.cc:493] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.348827 26699 raft_consensus.cc:515] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.351354 26699 leader_election.cc:290] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:34.352030 26668 consensus_queue.cc:260] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.352274 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.353173 26396 raft_consensus.cc:2393] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:34.353024 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c141b67bfa314acca1c0c3c29d3eaac8" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:34.354681 26512 leader_election.cc:304] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:34.355417 26699 raft_consensus.cc:3060] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.355616 26668 ts_tablet_manager.cc:1434] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 12:20:34.359143 26670 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:34.359695 26670 ts_tablet_manager.cc:1403] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.375s	user 0.055s	sys 0.146s
I20260709 12:20:34.362635 26670 raft_consensus.cc:359] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.363374 26670 raft_consensus.cc:385] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.363633 26670 raft_consensus.cc:740] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.364030 26699 raft_consensus.cc:2749] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:34.365959 26670 consensus_queue.cc:260] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.368933 26670 ts_tablet_manager.cc:1434] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:20:34.370002 26670 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:34.377307 26669 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:34.377873 26669 ts_tablet_manager.cc:1403] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.087s	user 0.008s	sys 0.002s
I20260709 12:20:34.379680 26670 tablet_bootstrap.cc:654] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.375433 26668 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:34.384397 26669 raft_consensus.cc:359] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.385085 26669 raft_consensus.cc:385] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.385366 26669 raft_consensus.cc:740] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.386231 26669 consensus_queue.cc:260] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.387599 26668 tablet_bootstrap.cc:654] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.385480 26667 tablet_bootstrap.cc:654] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.408951 26667 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.409476 26667 ts_tablet_manager.cc:1403] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.003s
I20260709 12:20:34.412636 26667 raft_consensus.cc:359] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.413157 26852 raft_consensus.cc:493] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.413376 26667 raft_consensus.cc:385] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.414407 26669 ts_tablet_manager.cc:1434] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 12:20:34.413785 26667 raft_consensus.cc:740] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.413777 26852 raft_consensus.cc:515] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.417234 26852 leader_election.cc:290] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:34.418469 26294 catalog_manager.cc:5697] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:34.420413 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.421113 26396 raft_consensus.cc:2468] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:34.421525 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:34.422417 26586 leader_election.cc:304] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
W20260709 12:20:34.423058 26588 leader_election.cc:343] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.423360 26852 raft_consensus.cc:2804] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.423821 26852 raft_consensus.cc:493] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.424419 26852 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.420269 26667 consensus_queue.cc:260] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.424851 26668 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:34.425380 26668 ts_tablet_manager.cc:1403] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.009s
I20260709 12:20:34.428299 26668 raft_consensus.cc:359] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.429406 26668 raft_consensus.cc:385] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.435372 26667 ts_tablet_manager.cc:1434] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:20:34.436440 26670 tablet_bootstrap.cc:492] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:34.445449 26668 raft_consensus.cc:740] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.438256 26852 raft_consensus.cc:515] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.445894 26670 ts_tablet_manager.cc:1403] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.076s	user 0.028s	sys 0.002s
I20260709 12:20:34.448570 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:34.449213 26396 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.450701 26667 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:34.448905 26670 raft_consensus.cc:359] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.453159 26670 raft_consensus.cc:385] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.453454 26670 raft_consensus.cc:740] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.454118 26670 consensus_queue.cc:260] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.457062 26667 tablet_bootstrap.cc:654] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.459388 26670 ts_tablet_manager.cc:1434] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:20:34.461964 26722 consensus_queue.cc:1048] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.462302 26396 raft_consensus.cc:2468] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:34.463873 26670 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:34.464423 26586 leader_election.cc:304] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.465387 26919 raft_consensus.cc:2804] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.465260 26668 consensus_queue.cc:260] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.445122 26669 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:34.465935 26919 raft_consensus.cc:697] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:34.471683 26919 consensus_queue.cc:237] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.476917 26786 raft_consensus.cc:493] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.477043 26731 consensus_queue.cc:1048] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:34.478609 26668 ts_tablet_manager.cc:1434] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.002s
I20260709 12:20:34.470065 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:34.479801 26668 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
W20260709 12:20:34.480903 26588 leader_election.cc:343] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.482244 26699 raft_consensus.cc:493] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.482758 26699 raft_consensus.cc:515] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.485477 26668 tablet_bootstrap.cc:654] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.485651 26699 leader_election.cc:290] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:34.488468 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:34.490947 26514 leader_election.cc:343] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.477475 26786 raft_consensus.cc:515] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.493191 26786 leader_election.cc:290] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:34.496794 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.498102 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.499020 26619 raft_consensus.cc:2468] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:34.501024 26667 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.501596 26667 ts_tablet_manager.cc:1403] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.004s
I20260709 12:20:34.504274 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.504983 26547 raft_consensus.cc:2468] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:34.467195 26852 leader_election.cc:290] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:34.505019 26667 raft_consensus.cc:359] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.505973 26667 raft_consensus.cc:385] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:34.507225 26513 leader_election.cc:343] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.507869 26513 leader_election.cc:304] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:34.509475 26848 raft_consensus.cc:2749] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:34.507411 26667 raft_consensus.cc:740] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.512725 26667 consensus_queue.cc:260] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.510839 26439 leader_election.cc:304] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:34.515053 26787 raft_consensus.cc:2804] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.515549 26787 raft_consensus.cc:493] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.515978 26787 raft_consensus.cc:3060] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.517560 26667 ts_tablet_manager.cc:1434] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:20:34.518687 26667 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:34.525863 26667 tablet_bootstrap.cc:654] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.470791 26670 tablet_bootstrap.cc:654] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.554191 26669 tablet_bootstrap.cc:654] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.554798 26296 catalog_manager.cc:5697] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:34.565888 26722 consensus_queue.cc:1048] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:34.613332 26667 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.614107 26667 ts_tablet_manager.cc:1403] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.096s	user 0.008s	sys 0.010s
I20260709 12:20:34.624701 26667 raft_consensus.cc:359] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.625494 26667 raft_consensus.cc:385] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.625865 26667 raft_consensus.cc:740] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.626984 26667 consensus_queue.cc:260] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.590912 26787 raft_consensus.cc:515] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.644860 26668 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:34.645555 26668 ts_tablet_manager.cc:1403] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.166s	user 0.055s	sys 0.070s
I20260709 12:20:34.647459 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:34.648136 26619 raft_consensus.cc:3060] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.649204 26668 raft_consensus.cc:359] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.649956 26668 raft_consensus.cc:385] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.650228 26668 raft_consensus.cc:740] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.650435 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9edfeb05dc4d379a4f87f500ab5f26" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:34.651088 26545 raft_consensus.cc:3060] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.650969 26668 consensus_queue.cc:260] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.656716 26668 ts_tablet_manager.cc:1434] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:20:34.658005 26668 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:34.660462 26667 ts_tablet_manager.cc:1434] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:20:34.660825 26787 leader_election.cc:290] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:34.665053 26668 tablet_bootstrap.cc:654] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.668653 26667 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:34.669570 26366 consensus_peers.cc:597] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:34.680855 26545 raft_consensus.cc:2468] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:34.681628 26910 raft_consensus.cc:493] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.682123 26910 raft_consensus.cc:515] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.683259 26670 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:34.683537 26911 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.683733 26670 ts_tablet_manager.cc:1403] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.000s
I20260709 12:20:34.684080 26911 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.686213 26911 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:34.686722 26670 raft_consensus.cc:359] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.687471 26670 raft_consensus.cc:385] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.687732 26670 raft_consensus.cc:740] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.688050 26915 raft_consensus.cc:493] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.688189 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.688392 26670 consensus_queue.cc:260] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.688550 26915 raft_consensus.cc:515] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.689369 26618 raft_consensus.cc:2468] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:34.690945 26364 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:34.692687 26722 raft_consensus.cc:2804] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:34.693010 26722 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.693050 26670 ts_tablet_manager.cc:1434] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:34.693365 26722 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.695755 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.696069 26620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.697561 26910 leader_election.cc:290] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:34.697820 26670 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
W20260709 12:20:34.698750 26364 leader_election.cc:343] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.691123 26915 leader_election.cc:290] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
W20260709 12:20:34.697659 26439 leader_election.cc:343] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.703519 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:34.707005 26441 leader_election.cc:343] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.707479 26441 leader_election.cc:304] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:34.710642 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.711493 26722 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.713150 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:34.713840 26722 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:34.712210 26915 raft_consensus.cc:2749] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:34.714850 26366 leader_election.cc:343] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:34.716756 26365 leader_election.cc:343] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.717234 26365 leader_election.cc:304] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:34.720785 26669 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:34.721297 26669 ts_tablet_manager.cc:1403] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.276s	user 0.024s	sys 0.011s
I20260709 12:20:34.721931 26620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:34.722596 26620 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.720796 26441 leader_election.cc:304] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:34.724268 26731 raft_consensus.cc:2804] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.724737 26731 raft_consensus.cc:697] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:34.725072 26619 raft_consensus.cc:2468] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:34.725766 26731 consensus_queue.cc:237] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:34.712046 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 143530us
I20260709 12:20:34.723471 26722 raft_consensus.cc:2749] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:34.733647 26667 tablet_bootstrap.cc:654] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.737613 26669 raft_consensus.cc:359] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.738526 26669 raft_consensus.cc:385] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.741179 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:34.738888 26669 raft_consensus.cc:740] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.745606 26670 tablet_bootstrap.cc:654] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:34.746327 26366 leader_election.cc:343] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.749948 26669 consensus_queue.cc:260] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.757419 26669 ts_tablet_manager.cc:1434] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 12:20:34.752187 26786 consensus_queue.cc:1048] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:20:34.766706 26669 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:34.757205 26787 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.767988 26668 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:34.767733 26787 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.768661 26668 ts_tablet_manager.cc:1403] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.046s
I20260709 12:20:34.769992 26787 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:34.846135 26699 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.846731 26699 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.864326 26667 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:34.864943 26667 ts_tablet_manager.cc:1403] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.197s	user 0.015s	sys 0.010s
I20260709 12:20:34.866539 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.869553 26620 raft_consensus.cc:2468] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:34.880635 26296 catalog_manager.cc:5697] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:34.880108 26668 raft_consensus.cc:359] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.902626 26668 raft_consensus.cc:385] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.905202 26910 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.905553 26699 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:34.908321 26670 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:34.905509 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:34.909137 26670 ts_tablet_manager.cc:1403] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.212s	user 0.018s	sys 0.019s
I20260709 12:20:34.906968 26668 raft_consensus.cc:740] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.909757 26786 raft_consensus.cc:493] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.914404 26786 raft_consensus.cc:515] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.916548 26786 leader_election.cc:290] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:34.919335 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.909991 26910 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.920830 26512 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:34.922191 26910 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:34.922748 26364 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:34.924263 26722 raft_consensus.cc:2804] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:34.923362 26699 raft_consensus.cc:2749] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:34.913491 26668 consensus_queue.cc:260] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:34.902709 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.925709 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:34.939678 26668 ts_tablet_manager.cc:1434] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.171s	user 0.015s	sys 0.046s
I20260709 12:20:34.940944 26668 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
W20260709 12:20:34.939426 26366 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.953789 26667 raft_consensus.cc:359] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.931039 26396 raft_consensus.cc:2468] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:34.956624 26667 raft_consensus.cc:385] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.957101 26667 raft_consensus.cc:740] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.958029 26722 raft_consensus.cc:697] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:34.958967 26667 consensus_queue.cc:260] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:34.959614 26722 consensus_queue.cc:237] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:34.960906 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
W20260709 12:20:34.956990 26441 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:34.967692 26667 ts_tablet_manager.cc:1434] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 12:20:34.969218 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:34.970079 26397 raft_consensus.cc:2393] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:34.972610 26668 tablet_bootstrap.cc:654] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:34.977349 26473 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:34.995940 26670 raft_consensus.cc:359] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:34.996747 26670 raft_consensus.cc:385] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:34.997028 26670 raft_consensus.cc:740] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:34.999425 26670 consensus_queue.cc:260] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.020170 26667 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:35.029313 26999 consensus_queue.cc:1048] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:35.055425 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:35.056457 26475 raft_consensus.cc:2468] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:35.064321 26670 ts_tablet_manager.cc:1434] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.155s	user 0.005s	sys 0.003s
I20260709 12:20:35.063133 26619 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.081282 26669 tablet_bootstrap.cc:654] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.083010 26670 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:35.099500 26919 consensus_queue.cc:1048] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:35.104724 26668 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:35.105628 26668 ts_tablet_manager.cc:1403] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.165s	user 0.010s	sys 0.010s
I20260709 12:20:35.109616 26668 raft_consensus.cc:359] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.110391 26668 raft_consensus.cc:385] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.110795 26668 raft_consensus.cc:740] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
W20260709 12:20:35.070897 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 84072us
I20260709 12:20:35.111725 26668 consensus_queue.cc:260] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.117043 26668 ts_tablet_manager.cc:1434] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
W20260709 12:20:35.117555 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:35.117858 26667 tablet_bootstrap.cc:654] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.106801 26296 catalog_manager.cc:5697] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:35.122864 26438 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:35.112875 27004 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.124617 26731 raft_consensus.cc:2804] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:35.127480 26731 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.127869 26731 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.124828 26438 leader_election.cc:304] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:35.129077 27004 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.135051 27004 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.136518 26731 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.139681 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:35.141062 26441 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.142277 26668 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:35.145418 27005 raft_consensus.cc:2749] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:35.135874 26365 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:35.145207 26731 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.162549 27004 raft_consensus.cc:2804] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:35.162998 27002 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.167066 27004 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.167512 27004 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.167286 27002 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.163407 26915 consensus_queue.cc:1048] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:35.173977 27002 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:35.181110 26731 consensus_queue.cc:1048] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:35.186269 26670 tablet_bootstrap.cc:654] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.195006 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.195529 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:35.197582 27002 consensus_queue.cc:1048] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:35.201449 26669 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:35.201875 26669 ts_tablet_manager.cc:1403] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.435s	user 0.013s	sys 0.019s
I20260709 12:20:35.202980 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:35.202891 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:35.205549 26473 raft_consensus.cc:2468] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:35.205940 26669 raft_consensus.cc:359] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.206887 26669 raft_consensus.cc:385] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.207808 26669 raft_consensus.cc:740] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.208894 26669 consensus_queue.cc:260] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.211848 27004 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.214259 27004 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.216259 26365 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:35.216915 26397 raft_consensus.cc:2393] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:35.217024 26910 raft_consensus.cc:2804] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:35.217613 26910 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:35.218518 26366 leader_election.cc:343] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.220973 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:35.221611 26398 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:35.217954 26910 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.223018 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:35.223726 26545 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.237381 26669 ts_tablet_manager.cc:1434] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 12:20:35.249423 26586 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:35.249799 27006 raft_consensus.cc:493] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.250324 27006 raft_consensus.cc:515] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.253119 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.253702 26548 raft_consensus.cc:2468] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:35.256690 26441 leader_election.cc:304] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:35.257297 26669 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:35.263672 26787 raft_consensus.cc:2804] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:35.264209 26787 raft_consensus.cc:493] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.264587 26787 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.289072 26668 tablet_bootstrap.cc:654] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.291033 27006 leader_election.cc:290] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:35.291507 26667 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:35.294427 26786 consensus_queue.cc:1048] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:35.274860 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:35.296384 26475 raft_consensus.cc:2393] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:35.298979 26915 consensus_queue.cc:1048] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:35.311470 26545 raft_consensus.cc:2468] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:35.312089 26667 ts_tablet_manager.cc:1403] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.298s	user 0.009s	sys 0.032s
I20260709 12:20:35.259676 26910 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.315912 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:35.332963 26787 raft_consensus.cc:515] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.346781 26787 leader_election.cc:290] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:35.360456 26667 raft_consensus.cc:359] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:35.361449 26667 raft_consensus.cc:385] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.362831 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
W20260709 12:20:35.363929 26439 leader_election.cc:343] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.367612 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
W20260709 12:20:35.369768 27011 log.cc:927] Time spent T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:20:35.385380 26910 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.387534 26667 raft_consensus.cc:740] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
W20260709 12:20:35.341636 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 48133.6us
I20260709 12:20:35.392829 26545 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.401827 26667 consensus_queue.cc:260] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:35.412314 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:35.412978 26475 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.417438 26438 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:35.421262 27020 raft_consensus.cc:2749] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:20:35.422073 26439 leader_election.cc:343] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.418844 26366 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
W20260709 12:20:35.426275 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 74654.4us
I20260709 12:20:35.432147 26668 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:35.433001 26668 ts_tablet_manager.cc:1403] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.315s	user 0.024s	sys 0.002s
I20260709 12:20:35.422726 26545 raft_consensus.cc:2468] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:35.456774 26669 tablet_bootstrap.cc:654] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.457011 26441 leader_election.cc:304] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:35.459151 26787 raft_consensus.cc:2804] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:35.459661 26787 raft_consensus.cc:697] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:35.460415 26787 consensus_queue.cc:237] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.461911 26668 raft_consensus.cc:359] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.463431 26668 raft_consensus.cc:385] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.435895 26852 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:35.465478 26852 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:35.465909 26852 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.450302 26475 raft_consensus.cc:2468] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:35.462435 26670 tablet_bootstrap.cc:492] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:35.468748 26670 ts_tablet_manager.cc:1403] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.395s	user 0.015s	sys 0.010s
I20260709 12:20:35.467728 26667 ts_tablet_manager.cc:1434] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.139s	user 0.008s	sys 0.000s
I20260709 12:20:35.437758 26722 raft_consensus.cc:2804] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:35.463779 26668 raft_consensus.cc:740] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.481691 26668 consensus_queue.cc:260] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.486847 26365 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:35.500852 26668 ts_tablet_manager.cc:1434] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.001s
I20260709 12:20:35.504334 26722 raft_consensus.cc:697] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:35.505252 26722 consensus_queue.cc:237] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.512458 26667 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:35.514667 27024 raft_consensus.cc:493] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.513942 26545 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.515280 27024 raft_consensus.cc:515] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:35.517547 27024 leader_election.cc:290] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:35.520262 26670 raft_consensus.cc:359] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.520994 26670 raft_consensus.cc:385] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.521253 26670 raft_consensus.cc:740] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.521934 27022 raft_consensus.cc:2804] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:35.522477 27022 raft_consensus.cc:697] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:35.522740 26668 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:35.523401 27022 consensus_queue.cc:237] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.524744 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:35.513616 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:35.538326 26852 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:35.542313 26364 leader_election.cc:343] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.537524 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.544185 26670 consensus_queue.cc:260] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.552348 26670 ts_tablet_manager.cc:1434] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 12:20:35.553385 26670 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
W20260709 12:20:35.563271 26366 leader_election.cc:343] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:35.564778 26366 consensus_peers.cc:597] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:35.565855 26366 leader_election.cc:343] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.566334 26366 leader_election.cc:304] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:35.567126 26667 tablet_bootstrap.cc:654] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.574191 27000 consensus_queue.cc:1048] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:35.577392 26668 tablet_bootstrap.cc:654] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.578122 26852 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:35.586429 27004 raft_consensus.cc:2749] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:35.588625 26295 catalog_manager.cc:5697] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:35.600677 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:35.630216 26669 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:35.630733 26669 ts_tablet_manager.cc:1403] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.383s	user 0.021s	sys 0.024s
I20260709 12:20:35.634316 26669 raft_consensus.cc:359] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.634994 26669 raft_consensus.cc:385] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.635273 26669 raft_consensus.cc:740] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.641714 26295 catalog_manager.cc:5697] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:35.640198 26669 consensus_queue.cc:260] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.653440 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:35.648762 26295 catalog_manager.cc:5697] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:35.687850 26669 ts_tablet_manager.cc:1434] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:20:35.718307 26670 tablet_bootstrap.cc:654] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.730248 26787 raft_consensus.cc:493] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.730795 26787 raft_consensus.cc:515] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.732770 26787 leader_election.cc:290] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:35.738546 26669 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:35.740913 26620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:35.741055 26622 raft_consensus.cc:1240] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Rejecting Update request from peer bb83296e641848deaddf424cc59fda0e for earlier term 1. Current term is 2. Ops: []
W20260709 12:20:35.745208 27033 log.cc:927] Time spent T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:20:35.745692 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:35.746443 26398 raft_consensus.cc:2468] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
W20260709 12:20:35.747502 26439 leader_election.cc:343] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.748356 26438 leader_election.cc:304] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:35.752089 26787 raft_consensus.cc:2804] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:35.752131 26667 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:35.752606 26787 raft_consensus.cc:493] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.752888 26667 ts_tablet_manager.cc:1403] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.254s	user 0.024s	sys 0.023s
I20260709 12:20:35.752959 26787 raft_consensus.cc:3060] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.750120 26915 consensus_queue.cc:1059] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:20:35.756140 26915 raft_consensus.cc:3055] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:35.757500 26915 raft_consensus.cc:740] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:35.760454 26668 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:35.760946 26668 ts_tablet_manager.cc:1403] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.239s	user 0.020s	sys 0.033s
I20260709 12:20:35.762421 26667 raft_consensus.cc:359] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.763106 26667 raft_consensus.cc:385] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.763401 26667 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.758469 26915 consensus_queue.cc:260] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.765473 26915 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:35.765528 26668 raft_consensus.cc:359] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.788079 26668 raft_consensus.cc:385] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.788380 26668 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.788962 26668 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.790668 26668 ts_tablet_manager.cc:1434] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 12:20:35.792487 26668 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:35.798275 26668 tablet_bootstrap.cc:654] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.775815 26787 raft_consensus.cc:515] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.804811 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
W20260709 12:20:35.806144 26439 leader_election.cc:343] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.809183 26787 leader_election.cc:290] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:35.764096 26667 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.811510 26852 raft_consensus.cc:493] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.812059 26852 raft_consensus.cc:515] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.813970 26852 leader_election.cc:290] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.814608 26667 ts_tablet_manager.cc:1434] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:20:35.815373 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.816008 26667 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:35.816176 26545 raft_consensus.cc:2393] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:35.820102 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:35.832337 26586 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:35.834000 26852 raft_consensus.cc:2749] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:35.826166 26588 leader_election.cc:304] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:35.839708 26669 tablet_bootstrap.cc:654] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.847102 26852 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.864373 26667 tablet_bootstrap.cc:654] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.847950 26670 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:35.848378 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:35.873023 26670 ts_tablet_manager.cc:1403] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.320s	user 0.011s	sys 0.002s
I20260709 12:20:35.873082 26399 raft_consensus.cc:3060] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.874408 26852 raft_consensus.cc:2749] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:35.877398 26670 raft_consensus.cc:359] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.884711 26399 raft_consensus.cc:2468] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:35.888952 26438 leader_election.cc:304] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:35.887933 26670 raft_consensus.cc:385] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.889580 26668 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:35.889766 26670 raft_consensus.cc:740] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.890142 26668 ts_tablet_manager.cc:1403] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.098s	user 0.027s	sys 0.055s
I20260709 12:20:35.890175 26787 raft_consensus.cc:2804] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:35.890776 26787 raft_consensus.cc:697] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:35.891546 26787 consensus_queue.cc:237] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.893097 26668 raft_consensus.cc:359] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.893777 26668 raft_consensus.cc:385] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.894042 26668 raft_consensus.cc:740] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.895833 26668 consensus_queue.cc:260] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.898094 26668 ts_tablet_manager.cc:1434] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:35.893961 26670 consensus_queue.cc:260] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.899132 26668 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:35.904204 26670 ts_tablet_manager.cc:1434] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.002s
I20260709 12:20:35.905033 26670 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:35.905937 26668 tablet_bootstrap.cc:654] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.911461 26722 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.912276 26669 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:35.912092 26722 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.912905 26669 ts_tablet_manager.cc:1403] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.176s	user 0.010s	sys 0.002s
I20260709 12:20:35.914601 26722 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.917721 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:35.918903 26365 leader_election.cc:343] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.918825 26669 raft_consensus.cc:359] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.919305 26699 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.919580 26669 raft_consensus.cc:385] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.919888 26669 raft_consensus.cc:740] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.919958 26915 raft_consensus.cc:493] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.920527 26915 raft_consensus.cc:515] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.920717 26669 consensus_queue.cc:260] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:35.921321 26670 tablet_bootstrap.cc:654] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.922510 26915 leader_election.cc:290] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:35.919777 26699 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.924888 26699 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:35.927057 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:35.927877 26472 raft_consensus.cc:2393] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:35.928154 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:35.929826 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:35.930621 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.931996 26512 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
W20260709 12:20:35.933171 26439 leader_election.cc:343] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.934171 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:35.937268 26668 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:35.938033 26668 ts_tablet_manager.cc:1403] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.007s
I20260709 12:20:35.942028 26668 raft_consensus.cc:359] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.942421 26998 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.942708 26668 raft_consensus.cc:385] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.943117 26668 raft_consensus.cc:740] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.944188 26667 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:35.943958 26668 consensus_queue.cc:260] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.944658 26667 ts_tablet_manager.cc:1403] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.000s
I20260709 12:20:35.946447 26668 ts_tablet_manager.cc:1434] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:35.947503 26668 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:35.947441 26667 raft_consensus.cc:359] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:35.948228 26667 raft_consensus.cc:385] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.948534 26667 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.949200 26667 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:35.950310 26441 leader_election.cc:343] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.950963 26441 leader_election.cc:304] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:35.952464 26787 raft_consensus.cc:2749] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:35.953989 26366 leader_election.cc:343] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.954614 26366 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:35.955250 26668 tablet_bootstrap.cc:654] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.956952 26722 raft_consensus.cc:2749] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:35.959303 27003 consensus_queue.cc:1048] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:35.959889 26296 catalog_manager.cc:5697] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:35.990267 26915 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.995862 26915 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:35.998309 26915 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:35.991595 27020 consensus_queue.cc:1048] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:20:36.000240 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.000609 26722 consensus_queue.cc:1048] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:36.002774 26438 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:35.986678 26667 ts_tablet_manager.cc:1434] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.001s
I20260709 12:20:36.007660 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:36.010850 26441 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.011315 26441 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:36.012859 26915 raft_consensus.cc:2749] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.034451 26667 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:36.048370 27000 consensus_queue.cc:1048] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:36.065941 27020 consensus_queue.cc:1048] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:36.070158 26667 tablet_bootstrap.cc:654] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.071841 26998 raft_consensus.cc:2749] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.073019 26669 ts_tablet_manager.cc:1434] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.160s	user 0.007s	sys 0.000s
I20260709 12:20:36.074180 26669 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:36.084658 26669 tablet_bootstrap.cc:654] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.093495 26670 tablet_bootstrap.cc:492] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.094038 26670 ts_tablet_manager.cc:1403] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.189s	user 0.049s	sys 0.101s
I20260709 12:20:36.097028 26670 raft_consensus.cc:359] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.098738 26852 raft_consensus.cc:493] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.101052 26852 raft_consensus.cc:515] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.103116 26852 leader_election.cc:290] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:36.104085 26670 raft_consensus.cc:385] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.104451 26670 raft_consensus.cc:740] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.105193 26670 consensus_queue.cc:260] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.100592 26669 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:36.106462 26669 ts_tablet_manager.cc:1403] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 12:20:36.106887 26668 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:36.107517 26668 ts_tablet_manager.cc:1403] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.160s	user 0.034s	sys 0.042s
I20260709 12:20:36.105993 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:36.112826 26670 ts_tablet_manager.cc:1434] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:20:36.112311 26668 raft_consensus.cc:359] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.115422 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7cc64cd1a8348f68f5b9115f9205e40" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.116097 26668 raft_consensus.cc:385] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.116447 26668 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.116557 26398 raft_consensus.cc:2393] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:36.114343 27000 consensus_queue.cc:1048] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:36.116864 26670 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:36.118251 26668 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.119825 26668 ts_tablet_manager.cc:1434] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 12:20:36.120790 26668 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:36.120682 26586 leader_election.cc:304] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:36.121485 26852 raft_consensus.cc:3060] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.122318 26667 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:36.122797 26667 ts_tablet_manager.cc:1403] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.089s	user 0.010s	sys 0.005s
I20260709 12:20:36.115270 26669 raft_consensus.cc:359] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.127694 26669 raft_consensus.cc:385] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.127987 26669 raft_consensus.cc:740] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.127919 26667 raft_consensus.cc:359] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.128778 26667 raft_consensus.cc:385] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.128757 26669 consensus_queue.cc:260] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.129333 26667 raft_consensus.cc:740] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.130278 26667 consensus_queue.cc:260] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.131875 26669 ts_tablet_manager.cc:1434] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:20:36.127144 26668 tablet_bootstrap.cc:654] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.133261 26852 raft_consensus.cc:2749] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.135537 26669 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:36.145833 26669 tablet_bootstrap.cc:654] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.167955 26667 ts_tablet_manager.cc:1434] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 12:20:36.175928 26667 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:36.186213 26670 tablet_bootstrap.cc:654] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.186751 27020 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.187592 27020 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.191221 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.191782 27020 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.192198 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:36.193174 26439 leader_election.cc:343] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.197350 26398 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:36.202569 26438 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.203414 26915 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.203800 26915 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.204542 26915 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.214529 26915 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.217290 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
W20260709 12:20:36.218433 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:20:36.218565 26439 leader_election.cc:343] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.219674 26670 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.220198 26670 ts_tablet_manager.cc:1403] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.001s
I20260709 12:20:36.220451 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:36.221104 26399 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.222846 26670 raft_consensus.cc:359] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.223523 26670 raft_consensus.cc:385] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.223770 26670 raft_consensus.cc:740] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.224579 26670 consensus_queue.cc:260] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.227289 26669 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:36.227757 26669 ts_tablet_manager.cc:1403] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.092s	user 0.008s	sys 0.004s
I20260709 12:20:36.230250 26670 ts_tablet_manager.cc:1434] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:20:36.230603 26915 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.230154 26399 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:36.231421 26670 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:36.231899 26438 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.233286 26915 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.233763 26915 raft_consensus.cc:697] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:36.234534 26915 consensus_queue.cc:237] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.238626 26668 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:36.239122 26668 ts_tablet_manager.cc:1403] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.119s	user 0.034s	sys 0.035s
I20260709 12:20:36.241983 26668 raft_consensus.cc:359] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.242596 26668 raft_consensus.cc:385] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.242865 26668 raft_consensus.cc:740] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.243460 26668 consensus_queue.cc:260] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.243860 26998 raft_consensus.cc:493] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.244354 26998 raft_consensus.cc:515] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.245937 26668 ts_tablet_manager.cc:1434] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:36.246431 26998 leader_election.cc:290] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:36.246878 26670 tablet_bootstrap.cc:654] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.247920 26668 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:36.250101 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:36.250767 26472 raft_consensus.cc:2468] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:36.251772 26667 tablet_bootstrap.cc:654] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.252400 26514 leader_election.cc:304] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:36.253667 26668 tablet_bootstrap.cc:654] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.254354 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.254951 26998 raft_consensus.cc:2804] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.255010 26619 raft_consensus.cc:2468] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:36.260710 26998 raft_consensus.cc:493] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.261132 26998 raft_consensus.cc:3060] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.261754 26670 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.262238 26670 ts_tablet_manager.cc:1403] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.031s	user 0.001s	sys 0.009s
I20260709 12:20:36.264518 26699 raft_consensus.cc:493] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.259855 26296 catalog_manager.cc:5697] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.252246 26669 raft_consensus.cc:359] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.264652 26670 raft_consensus.cc:359] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.265213 26669 raft_consensus.cc:385] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.265331 26670 raft_consensus.cc:385] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.265630 26669 raft_consensus.cc:740] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.265717 26670 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.264998 26699 raft_consensus.cc:515] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.268590 26699 leader_election.cc:290] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.266593 26670 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.270140 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.270439 27003 raft_consensus.cc:493] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.270985 26398 raft_consensus.cc:2468] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:36.271134 27003 raft_consensus.cc:515] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.272521 26512 leader_election.cc:304] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:36.273244 26699 raft_consensus.cc:2804] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.273679 26699 raft_consensus.cc:493] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.274202 26699 raft_consensus.cc:3060] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.274444 27003 leader_election.cc:290] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:36.277370 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.266623 26669 consensus_queue.cc:260] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:36.279125 26364 leader_election.cc:343] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.273614 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.280118 26669 ts_tablet_manager.cc:1434] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
W20260709 12:20:36.280701 26513 leader_election.cc:343] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.281169 26669 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:36.282137 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.282366 26668 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:36.282968 26668 ts_tablet_manager.cc:1403] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.004s
I20260709 12:20:36.285454 26998 raft_consensus.cc:515] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.283267 26670 ts_tablet_manager.cc:1434] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 12:20:36.283490 26366 leader_election.cc:343] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.288203 26366 leader_election.cc:304] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.288456 26670 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:36.289377 27003 raft_consensus.cc:2749] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.284061 26699 raft_consensus.cc:515] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.290613 26472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:36.291203 26472 raft_consensus.cc:3060] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.298403 26670 tablet_bootstrap.cc:654] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.299499 26667 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:36.300014 26667 ts_tablet_manager.cc:1403] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.124s	user 0.015s	sys 0.004s
I20260709 12:20:36.302428 26667 raft_consensus.cc:359] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.303007 26667 raft_consensus.cc:385] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.303267 26667 raft_consensus.cc:740] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.303895 26667 consensus_queue.cc:260] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.305310 26667 ts_tablet_manager.cc:1434] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:20:36.310160 26667 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:36.309885 26915 consensus_queue.cc:1048] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:36.312256 26699 leader_election.cc:290] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.314800 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:36.315436 26397 raft_consensus.cc:3060] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.287000 26668 raft_consensus.cc:359] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.319698 26668 raft_consensus.cc:385] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.320060 26668 raft_consensus.cc:740] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.320320 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:36.320742 26668 consensus_queue.cc:260] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:36.322522 26513 leader_election.cc:343] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.326325 26397 raft_consensus.cc:2468] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:36.327945 27020 consensus_queue.cc:1048] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:36.330549 26998 leader_election.cc:290] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:36.330741 26512 leader_election.cc:304] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.321208 26669 tablet_bootstrap.cc:654] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.333076 26998 raft_consensus.cc:2804] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.334816 26998 raft_consensus.cc:697] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:36.335631 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "050c5f3e50244a45886a0ee8245165af" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:36.336628 26619 raft_consensus.cc:3060] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.341702 26670 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.342294 26670 ts_tablet_manager.cc:1403] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.005s
I20260709 12:20:36.345522 26668 ts_tablet_manager.cc:1434] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.000s
I20260709 12:20:36.345306 26670 raft_consensus.cc:359] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.346236 26670 raft_consensus.cc:385] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.346585 26670 raft_consensus.cc:740] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.347343 26670 consensus_queue.cc:260] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.349092 26670 ts_tablet_manager.cc:1434] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:36.351761 26670 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:36.358024 26670 tablet_bootstrap.cc:654] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.360862 26668 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:36.368732 26667 tablet_bootstrap.cc:654] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.381834 26668 tablet_bootstrap.cc:654] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.391360 26670 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.392055 26670 ts_tablet_manager.cc:1403] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.007s
I20260709 12:20:36.395274 26472 raft_consensus.cc:2468] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:36.396689 26699 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.397058 26670 raft_consensus.cc:359] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.397321 26699 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.398748 26670 raft_consensus.cc:385] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.399152 26670 raft_consensus.cc:740] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.399932 26670 consensus_queue.cc:260] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.401054 26699 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.403219 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.405682 26670 ts_tablet_manager.cc:1434] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:20:36.407878 26852 raft_consensus.cc:493] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.407521 26514 leader_election.cc:304] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:36.409847 26699 raft_consensus.cc:2804] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.410466 26668 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:36.411000 26668 ts_tablet_manager.cc:1403] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 12:20:36.411523 26699 raft_consensus.cc:697] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:36.408465 26852 raft_consensus.cc:515] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.418697 26668 raft_consensus.cc:359] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.419494 26668 raft_consensus.cc:385] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.419840 26668 raft_consensus.cc:740] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.412510 26699 consensus_queue.cc:237] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.421298 26852 leader_election.cc:290] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.423857 26667 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:36.423831 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:36.424728 26667 ts_tablet_manager.cc:1403] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.115s	user 0.017s	sys 0.002s
I20260709 12:20:36.424816 26475 raft_consensus.cc:2468] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:36.421844 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:36.426796 26589 leader_election.cc:343] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.428354 26667 raft_consensus.cc:359] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.429124 26667 raft_consensus.cc:385] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.429737 27020 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.430506 27020 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.429384 26588 leader_election.cc:304] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:36.436360 26852 raft_consensus.cc:2804] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.436769 26852 raft_consensus.cc:493] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.438875 26852 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.439906 27020 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.458169 26915 raft_consensus.cc:493] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.458797 26915 raft_consensus.cc:515] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.335678 26998 consensus_queue.cc:237] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.482292 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.480214 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.486723 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.488320 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.489089 26397 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:36.494488 26670 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
W20260709 12:20:36.499500 26441 leader_election.cc:343] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.497874 26915 leader_election.cc:290] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:36.421871 26668 consensus_queue.cc:260] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.506605 26668 ts_tablet_manager.cc:1434] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.000s
I20260709 12:20:36.517829 26668 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:36.429380 26667 raft_consensus.cc:740] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.507668 26919 raft_consensus.cc:493] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.527947 26919 raft_consensus.cc:515] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:36.510520 26441 leader_election.cc:343] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.495149 26619 raft_consensus.cc:2468] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:36.530393 26438 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:36.537644 26731 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.538112 26731 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.538466 26731 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.546069 26731 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.530893 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.531191 26667 consensus_queue.cc:260] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.531751 26669 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:36.532645 26513 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:36.551520 26669 ts_tablet_manager.cc:1403] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.271s	user 0.035s	sys 0.045s
I20260709 12:20:36.548190 26731 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.549903 26670 tablet_bootstrap.cc:654] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.509436 26852 raft_consensus.cc:515] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.559353 26848 raft_consensus.cc:2749] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.555248 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:36.560774 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:36.561252 26397 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.563060 26667 ts_tablet_manager.cc:1434] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.138s	user 0.009s	sys 0.000s
I20260709 12:20:36.564082 26667 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:36.568166 26919 leader_election.cc:290] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.569365 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:36.569537 26439 leader_election.cc:343] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:36.569751 26441 leader_election.cc:343] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.570036 26439 leader_election.cc:304] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.570223 26399 raft_consensus.cc:2393] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate 297900273c7740af9938749b3ce3a80d in this term.
I20260709 12:20:36.571091 27020 raft_consensus.cc:2749] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.579988 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:36.580503 26668 tablet_bootstrap.cc:654] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.580704 26475 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.590787 26852 leader_election.cc:290] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.593402 26669 raft_consensus.cc:359] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.604758 26669 raft_consensus.cc:385] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.605644 26669 raft_consensus.cc:740] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.606431 26669 consensus_queue.cc:260] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.618932 26475 raft_consensus.cc:2468] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:36.621996 26588 leader_election.cc:304] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:36.625703 26852 raft_consensus.cc:2804] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.628794 26669 ts_tablet_manager.cc:1434] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.000s
I20260709 12:20:36.631845 26397 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:36.633401 26669 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:36.636726 26667 tablet_bootstrap.cc:654] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.639915 26438 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:36.641625 27073 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.639842 26298 catalog_manager.cc:5697] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:36.680462 26294 catalog_manager.cc:5697] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:36.645254 27073 raft_consensus.cc:697] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:36.690438 26668 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:36.691206 26668 ts_tablet_manager.cc:1403] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.174s	user 0.008s	sys 0.006s
W20260709 12:20:36.643046 26513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 110486us
W20260709 12:20:36.693845 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 47936.6us
W20260709 12:20:36.707145 27072 log.cc:927] Time spent T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:20:36.711035 27073 consensus_queue.cc:237] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.711905 27078 raft_consensus.cc:493] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.712529 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:36.712415 27078 raft_consensus.cc:515] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.713632 26670 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:36.714125 26670 ts_tablet_manager.cc:1403] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.228s	user 0.011s	sys 0.015s
I20260709 12:20:36.711117 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ee0ada749043888b2a134a520a25e1" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:36.718372 26589 leader_election.cc:343] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.718861 27005 raft_consensus.cc:493] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.718971 26622 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.719517 27005 raft_consensus.cc:515] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.721652 27005 leader_election.cc:290] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:36.722981 26669 tablet_bootstrap.cc:654] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.726101 27078 leader_election.cc:290] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.732738 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.738204 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.738843 26397 raft_consensus.cc:2468] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:36.740191 27077 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.740731 27077 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.744556 26670 raft_consensus.cc:359] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.747928 26670 raft_consensus.cc:385] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.750761 26670 raft_consensus.cc:740] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.747180 26589 leader_election.cc:304] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:36.745877 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
W20260709 12:20:36.751487 26441 leader_election.cc:343] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.746095 26547 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.764649 26852 raft_consensus.cc:697] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:36.752031 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.755776 26668 raft_consensus.cc:359] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.765590 26852 consensus_queue.cc:237] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.784075 26668 raft_consensus.cc:385] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.784572 26668 raft_consensus.cc:740] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.764559 26512 leader_election.cc:304] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:36.785847 26699 raft_consensus.cc:2804] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.765228 26670 consensus_queue.cc:260] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:36.786231 26699 raft_consensus.cc:493] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.786847 26699 raft_consensus.cc:3060] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.789649 26668 consensus_queue.cc:260] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.793910 27080 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.794421 27080 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:36.796096 26439 leader_election.cc:343] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.796550 26439 leader_election.cc:304] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:36.799078 26699 raft_consensus.cc:515] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:36.801069 26731 raft_consensus.cc:2749] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.798439 26915 raft_consensus.cc:493] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.805476 26699 leader_election.cc:290] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:36.805289 26915 raft_consensus.cc:515] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.807639 26667 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:36.808218 26667 ts_tablet_manager.cc:1403] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.244s	user 0.018s	sys 0.010s
I20260709 12:20:36.800724 27077 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
W20260709 12:20:36.809293 26513 leader_election.cc:343] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.834524 26667 raft_consensus.cc:359] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.835598 26667 raft_consensus.cc:385] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.835968 26667 raft_consensus.cc:740] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.851217 26668 ts_tablet_manager.cc:1434] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.160s	user 0.006s	sys 0.004s
I20260709 12:20:36.854509 26852 raft_consensus.cc:3060] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.857667 26668 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: Bootstrap starting.
I20260709 12:20:36.859346 26667 consensus_queue.cc:260] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.875456 26669 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
W20260709 12:20:36.883391 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 87325us
I20260709 12:20:36.884186 26669 ts_tablet_manager.cc:1403] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.251s	user 0.018s	sys 0.000s
I20260709 12:20:36.835384 27038 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:36.862174 26586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 118728us
I20260709 12:20:36.869405 26667 ts_tablet_manager.cc:1434] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.002s
I20260709 12:20:36.892748 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
W20260709 12:20:36.861099 26589 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 103536us
W20260709 12:20:36.864176 26513 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 57310.3us
I20260709 12:20:36.898380 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.875505 27020 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:20:36.902376 26298 catalog_manager.cc:5697] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:36.891188 27038 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:36.912955 26366 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.916507 26852 raft_consensus.cc:2749] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:36.918541 26669 raft_consensus.cc:359] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.919564 26669 raft_consensus.cc:385] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:36.920128 26669 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:36.922765 26670 ts_tablet_manager.cc:1434] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.208s	user 0.011s	sys 0.021s
I20260709 12:20:36.897651 27080 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.925932 26669 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.904366 27093 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.907248 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:36.928516 26621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:36.936401 27093 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.930366 26400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.937616 26618 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:36.931430 26915 leader_election.cc:290] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.917136 27073 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:20:36.935916 27085 raft_consensus.cc:493] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.941169 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:36.943449 27085 raft_consensus.cc:515] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:36.948418 27038 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:36.948362 26400 raft_consensus.cc:2468] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 0.
I20260709 12:20:36.949501 27085 leader_election.cc:290] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:36.950461 26396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.951340 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:36.951107 26396 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
W20260709 12:20:36.953685 26588 leader_election.cc:343] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:36.959510 26366 leader_election.cc:343] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:36.938381 26513 leader_election.cc:343] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.954645 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:36.938592 27093 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:36.955605 26667 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:36.961666 26668 tablet_bootstrap.cc:654] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.957561 26395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:36.964493 26364 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:36.965415 26586 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:36.968391 26699 consensus_queue.cc:1048] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:36.968945 27093 raft_consensus.cc:2804] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.969385 27093 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.969802 27093 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.967975 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:36.970377 26395 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:36.970743 26399 raft_consensus.cc:3060] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.971216 27085 raft_consensus.cc:2804] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:36.971647 27085 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.971983 27085 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:36.975734 26586 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: bb83296e641848deaddf424cc59fda0e
I20260709 12:20:36.977525 26669 ts_tablet_manager.cc:1434] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:20:36.977775 27088 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:36.978206 27088 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:36.978562 27088 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:36.978595 26669 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:36.979763 26295 catalog_manager.cc:5697] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:36.989037 27086 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:36.989535 27086 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:36.993857 27086 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:36.994825 26667 tablet_bootstrap.cc:654] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:36.998181 26365 leader_election.cc:343] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:36.999279 26399 raft_consensus.cc:2468] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:37.000741 26512 leader_election.cc:304] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:37.000800 26699 consensus_queue.cc:1048] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:36.991758 27093 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.002609 26670 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
W20260709 12:20:37.004230 26441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 71317.8us
I20260709 12:20:37.004711 27085 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.007421 26438 leader_election.cc:304] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:37.008189 27005 raft_consensus.cc:2804] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.008257 26919 raft_consensus.cc:493] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.008646 27005 raft_consensus.cc:493] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.009109 27005 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.009135 26919 raft_consensus.cc:515] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:37.010591 27093 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:37.017498 26699 raft_consensus.cc:2804] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.018257 26699 raft_consensus.cc:697] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:37.021502 26699 consensus_queue.cc:237] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.041944 26475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:37.042794 26475 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:36.991189 27088 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.032691 27085 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:37.045784 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:37.047694 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:37.046384 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:37.062983 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:37.076870 26623 raft_consensus.cc:1240] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Rejecting Update request from peer bb83296e641848deaddf424cc59fda0e for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:37.077428 26619 raft_consensus.cc:2393] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 2: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:37.060401 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:37.079149 26617 raft_consensus.cc:2393] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:37.077437 26392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:37.081847 26392 raft_consensus.cc:2393] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:37.082584 26998 consensus_queue.cc:1048] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:37.083457 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:37.084208 26471 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
W20260709 12:20:37.087466 26588 leader_election.cc:343] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.087903 26588 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:37.090384 26731 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.090988 26731 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.096421 27098 raft_consensus.cc:2749] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.097193 26669 tablet_bootstrap.cc:654] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.098158 26731 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:37.094310 27020 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.098975 27020 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:37.099833 26441 leader_election.cc:343] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.110707 26667 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:37.111503 26667 ts_tablet_manager.cc:1403] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.156s	user 0.021s	sys 0.011s
I20260709 12:20:37.115049 26365 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:37.118073 26731 consensus_queue.cc:1059] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:37.118885 26731 raft_consensus.cc:3055] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:37.119343 26731 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:37.120081 26731 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.125190 26731 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:37.100986 27020 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:37.136615 26365 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:37.135099 27003 raft_consensus.cc:2804] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.141578 27003 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.141991 27003 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.143603 26622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:37.144658 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:37.145349 26623 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:37.150014 27088 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:37.150674 27003 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:37.152907 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:37.153059 27003 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
W20260709 12:20:37.153517 26439 leader_election.cc:343] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.154464 26439 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:37.156975 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:37.158341 26474 raft_consensus.cc:2410] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:20:37.167683 26395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:37.168053 26392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:37.168905 26392 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:37.172897 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:37.172613 26998 consensus_queue.cc:1048] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:37.177234 26366 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.178543 27000 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:20:37.177714 26365 leader_election.cc:343] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.180042 26365 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:37.178999 27000 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:37.179064 26919 leader_election.cc:290] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:37.182855 27000 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:37.193346 26668 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: No bootstrap required, opened a new log
I20260709 12:20:37.193944 26668 ts_tablet_manager.cc:1403] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: Time spent bootstrapping tablet: real 0.337s	user 0.022s	sys 0.006s
I20260709 12:20:37.194125 26670 tablet_bootstrap.cc:654] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:37.214537 26438 leader_election.cc:343] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.215112 26438 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:37.215827 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:37.216467 27084 raft_consensus.cc:2749] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.216571 26471 raft_consensus.cc:2468] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:37.217216 27084 consensus_queue.cc:1048] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:37.217944 26667 raft_consensus.cc:359] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.218664 26667 raft_consensus.cc:385] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.218942 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:37.219725 26470 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.220075 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:37.207509 27022 raft_consensus.cc:2749] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.218993 26667 raft_consensus.cc:740] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.226963 26667 consensus_queue.cc:260] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.225849 26588 leader_election.cc:304] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:37.188252 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:37.236896 27038 raft_consensus.cc:2804] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.239733 26586 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:37.243068 27093 raft_consensus.cc:2749] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:37.238519 27038 raft_consensus.cc:493] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.244621 27038 raft_consensus.cc:3060] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.246892 26668 raft_consensus.cc:359] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.248088 26668 raft_consensus.cc:385] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.248385 26668 raft_consensus.cc:740] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.249111 26668 consensus_queue.cc:260] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.219738 27005 raft_consensus.cc:515] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:37.256484 26473 raft_consensus.cc:2393] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 1: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:37.258693 26667 ts_tablet_manager.cc:1434] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.147s	user 0.007s	sys 0.000s
I20260709 12:20:37.251654 26668 ts_tablet_manager.cc:1434] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:20:37.270335 27100 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:37.270931 27100 raft_consensus.cc:493] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.271431 27100 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:37.278847 27005 leader_election.cc:290] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
W20260709 12:20:37.262288 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 44006.3us
I20260709 12:20:37.298302 26548 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.305482 27100 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.309983 26731 consensus_queue.cc:1048] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:37.285081 26589 leader_election.cc:343] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.313069 26699 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.313612 26699 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:37.314996 26589 consensus_peers.cc:597] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.338476 26699 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:37.349296 26667 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:37.366724 26365 leader_election.cc:304] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:37.353819 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:37.374354 27081 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.375198 26395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:37.376011 26670 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:37.395982 26670 ts_tablet_manager.cc:1403] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.448s	user 0.012s	sys 0.016s
I20260709 12:20:37.366911 27000 raft_consensus.cc:515] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:37.382143 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 205414us
I20260709 12:20:37.404560 26667 tablet_bootstrap.cc:654] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.398340 26618 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:37.395601 27100 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:37.416208 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:37.425321 26670 raft_consensus.cc:359] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.426091 26670 raft_consensus.cc:385] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.426402 26670 raft_consensus.cc:740] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.505218 26670 consensus_queue.cc:260] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.507431 26669 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:37.507987 26669 ts_tablet_manager.cc:1403] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.530s	user 0.017s	sys 0.006s
I20260709 12:20:37.402070 26395 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
W20260709 12:20:37.531701 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:37.533424 26669 raft_consensus.cc:359] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.534363 26669 raft_consensus.cc:385] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.534684 26669 raft_consensus.cc:740] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.536105 26670 ts_tablet_manager.cc:1434] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.140s	user 0.005s	sys 0.002s
I20260709 12:20:37.537228 26670 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:37.536990 27000 leader_election.cc:290] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:37.543537 26669 consensus_queue.cc:260] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:37.547677 27106 log.cc:927] Time spent T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:20:37.549503 26669 ts_tablet_manager.cc:1434] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:20:37.551178 26669 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:37.566325 27038 raft_consensus.cc:515] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:37.568040 26670 tablet_bootstrap.cc:654] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.581797 26512 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:37.590392 26618 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 3.
I20260709 12:20:37.587117 26473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
W20260709 12:20:37.604321 26364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 101267us
I20260709 12:20:37.582227 27038 leader_election.cc:290] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:37.601707 26469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:37.616036 26473 raft_consensus.cc:2368] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:37.617885 26616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:37.622170 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:37.598242 26470 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:37.629768 26439 leader_election.cc:304] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:37.619124 26616 raft_consensus.cc:2368] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:37.639590 27121 raft_consensus.cc:2804] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:37.639911 27071 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:37.640175 27121 raft_consensus.cc:697] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:37.641270 27121 consensus_queue.cc:237] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.680527 26469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:37.681399 26469 raft_consensus.cc:3060] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.689457 26670 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:37.691355 27129 raft_consensus.cc:493] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.691929 27129 raft_consensus.cc:515] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.696322 27129 leader_election.cc:290] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:37.697687 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:37.699585 27081 raft_consensus.cc:2749] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.672187 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:37.703955 26667 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:37.704495 26667 ts_tablet_manager.cc:1403] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.407s	user 0.033s	sys 0.046s
W20260709 12:20:37.708132 26589 leader_election.cc:343] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.712230 26364 leader_election.cc:400] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Vote denied by peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181) with higher term. Message: Invalid argument: T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:37.712647 26364 leader_election.cc:403] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:20:37.713336 26364 leader_election.cc:343] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.691396 26670 ts_tablet_manager.cc:1403] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.154s	user 0.017s	sys 0.002s
I20260709 12:20:37.716207 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:37.716040 26667 raft_consensus.cc:359] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.716904 26667 raft_consensus.cc:385] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.717200 26667 raft_consensus.cc:740] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.717983 26669 tablet_bootstrap.cc:654] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.717932 26667 consensus_queue.cc:260] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:37.721282 26366 leader_election.cc:343] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.721722 26366 leader_election.cc:304] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:37.722497 26667 ts_tablet_manager.cc:1434] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 12:20:37.720957 26365 leader_election.cc:400] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Vote denied by peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205) with higher term. Message: Invalid argument: T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:37.730670 26667 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:37.712466 26469 raft_consensus.cc:2468] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:37.734649 26670 raft_consensus.cc:359] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.736368 26670 raft_consensus.cc:385] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.736652 26670 raft_consensus.cc:740] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.737520 26670 consensus_queue.cc:260] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.739631 27097 raft_consensus.cc:493] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.740135 27097 raft_consensus.cc:515] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.742281 27097 leader_election.cc:290] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:37.747839 27131 raft_consensus.cc:3060] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:37.751995 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:37.728602 26365 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:37.752853 26548 raft_consensus.cc:2468] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:37.748409 26615 raft_consensus.cc:3060] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.748543 26588 leader_election.cc:304] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:37.755371 27133 raft_consensus.cc:2804] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.757148 27133 raft_consensus.cc:697] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:37.757970 27071 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:37.758307 26589 leader_election.cc:304] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:37.758193 27133 consensus_queue.cc:237] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:37.760241 27038 raft_consensus.cc:2804] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.760658 27038 raft_consensus.cc:493] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.761044 27038 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.748991 26398 raft_consensus.cc:2468] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:37.749631 27125 raft_consensus.cc:2749] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.796761 26670 ts_tablet_manager.cc:1434] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.001s
I20260709 12:20:37.763705 26919 raft_consensus.cc:2804] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.771307 27071 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:37.813104 27038 raft_consensus.cc:515] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:37.815728 27038 leader_election.cc:290] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
W20260709 12:20:37.827059 26441 leader_election.cc:343] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:37.831343 26586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 61000.7us
I20260709 12:20:37.830855 26919 raft_consensus.cc:697] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:37.838954 26919 consensus_queue.cc:237] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:37.839856 27113 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.840441 27113 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.824344 26392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:37.844509 26397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:37.846648 26392 raft_consensus.cc:2468] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:37.842773 27113 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:37.846577 26667 tablet_bootstrap.cc:654] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.866067 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "966ed0b8df3447849f8bccb9b1af5b33" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:37.887603 26669 tablet_bootstrap.cc:492] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:37.888427 26669 ts_tablet_manager.cc:1403] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.337s	user 0.021s	sys 0.049s
I20260709 12:20:37.890278 26667 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: No bootstrap required, opened a new log
I20260709 12:20:37.890775 26667 ts_tablet_manager.cc:1403] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.167s	user 0.009s	sys 0.011s
I20260709 12:20:37.898212 27137 consensus_queue.cc:1048] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:20:37.936808 26669 raft_consensus.cc:359] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:37.937672 26669 raft_consensus.cc:385] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.938019 26669 raft_consensus.cc:740] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.938707 26669 consensus_queue.cc:260] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:37.941323 26667 raft_consensus.cc:359] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.942111 26667 raft_consensus.cc:385] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.942420 26667 raft_consensus.cc:740] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.943118 26667 consensus_queue.cc:260] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:37.943820 26669 ts_tablet_manager.cc:1434] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.002s
I20260709 12:20:37.948695 26699 consensus_queue.cc:1048] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:37.954288 26669 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:37.930759 26438 leader_election.cc:304] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:37.967370 27121 raft_consensus.cc:2804] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.967916 27121 raft_consensus.cc:697] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:37.968045 27131 raft_consensus.cc:2749] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181) with higher term. Message: Invalid argument: T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:37.968817 27121 consensus_queue.cc:237] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:37.982003 26667 ts_tablet_manager.cc:1434] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.091s	user 0.005s	sys 0.003s
I20260709 12:20:38.020516 26398 raft_consensus.cc:2468] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 3.
I20260709 12:20:38.025318 26669 tablet_bootstrap.cc:654] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.025611 27071 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:38.027041 26548 raft_consensus.cc:1240] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Rejecting Update request from peer bb83296e641848deaddf424cc59fda0e for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:38.045226 27098 raft_consensus.cc:493] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.045876 27098 raft_consensus.cc:515] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.048112 26848 consensus_queue.cc:1048] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.057085 26399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
W20260709 12:20:38.059458 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 142864us
I20260709 12:20:38.067307 27111 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.067962 27111 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.070573 27111 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:38.071771 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:38.053356 27071 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:38.076462 27078 raft_consensus.cc:493] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.088181 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:38.092378 26549 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.092603 26392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:38.093684 26392 raft_consensus.cc:2393] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:38.094452 26294 catalog_manager.cc:5697] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.094749 27078 raft_consensus.cc:515] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.098683 27078 leader_election.cc:290] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:38.094460 26514 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:38.101073 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.101281 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.102156 26617 raft_consensus.cc:2393] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:38.104343 26439 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:38.108115 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:38.108790 27078 raft_consensus.cc:2749] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:38.108894 26470 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 0.
I20260709 12:20:38.061209 26915 consensus_queue.cc:1059] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:20:38.119657 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcf2bebeaf8451e890bcbe46a6ae967" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:38.120409 26549 raft_consensus.cc:2468] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:38.125641 26471 raft_consensus.cc:2393] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:38.126950 26915 raft_consensus.cc:3055] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:38.128232 26915 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:38.128443 26619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.129222 26915 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:38.130425 26915 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:38.132696 26670 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:38.135856 26514 leader_election.cc:304] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:38.137629 27078 raft_consensus.cc:3060] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.117197 26365 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
W20260709 12:20:38.148110 26364 leader_election.cc:343] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.155436 27085 raft_consensus.cc:2804] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:38.158026 27085 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.158622 27085 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.144945 27084 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.161237 26589 leader_election.cc:304] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:38.162619 27098 leader_election.cc:290] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:38.163542 27098 raft_consensus.cc:2804] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:38.164402 27098 raft_consensus.cc:697] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:38.165594 27098 consensus_queue.cc:237] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.173142 27078 raft_consensus.cc:2749] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:38.174175 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:38.180621 26670 tablet_bootstrap.cc:654] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.182888 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f6c90924ba74445ae2edc110c400e78" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:38.196031 26623 raft_consensus.cc:1275] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:38.231691 26586 leader_election.cc:304] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:38.232861 27093 raft_consensus.cc:2749] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:38.236179 27145 log.cc:927] Time spent T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:20:38.236555 27084 raft_consensus.cc:2749] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:38.237722 27123 consensus_queue.cc:1048] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:20:38.242731 26669 tablet_bootstrap.cc:492] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:38.243292 26669 ts_tablet_manager.cc:1403] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.290s	user 0.022s	sys 0.031s
I20260709 12:20:38.255657 26669 raft_consensus.cc:359] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.257292 26669 raft_consensus.cc:385] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.257758 26669 raft_consensus.cc:740] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.264878 26669 consensus_queue.cc:260] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.276247 26670 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:38.277021 26670 ts_tablet_manager.cc:1403] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.145s	user 0.020s	sys 0.010s
I20260709 12:20:38.288522 26670 raft_consensus.cc:359] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.289234 26670 raft_consensus.cc:385] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.289521 26670 raft_consensus.cc:740] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.290396 26670 consensus_queue.cc:260] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.303864 27077 consensus_queue.cc:1048] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:38.318831 27073 raft_consensus.cc:493] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.322283 26919 consensus_queue.cc:1048] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.378690 26669 ts_tablet_manager.cc:1434] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.135s	user 0.008s	sys 0.001s
I20260709 12:20:38.367923 27137 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.388182 26669 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:38.374413 27073 raft_consensus.cc:515] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.391927 27142 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.406112 26393 raft_consensus.cc:1275] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:38.387959 27137 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.414923 27073 leader_election.cc:290] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:38.416541 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.417035 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:38.418310 26441 leader_election.cc:343] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.406524 27142 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:38.421420 26366 consensus_peers.cc:597] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:38.422370 26439 leader_election.cc:343] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.428378 27073 consensus_queue.cc:1048] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:20:38.436896 27142 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:38.411695 27085 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.438349 26398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:38.444401 27085 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:38.453876 26670 ts_tablet_manager.cc:1434] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.176s	user 0.008s	sys 0.003s
I20260709 12:20:38.454902 26670 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:38.458736 27137 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:38.461587 26439 leader_election.cc:304] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:38.467757 27073 raft_consensus.cc:2749] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:38.467927 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 49306.4us
W20260709 12:20:38.473248 26441 consensus_peers.cc:597] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:38.460178 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:38.474390 26392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:38.474521 26615 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.475469 26392 raft_consensus.cc:2393] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39930cd9790d4943bdec6f9df4ce2641 in current term 1: Already voted for candidate fc524c91d3c249c0aeb08dae83b98e41 in this term.
I20260709 12:20:38.467813 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:38.481912 27084 consensus_queue.cc:1048] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:38.486989 26471 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:38.489073 26469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:38.489696 26469 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.502656 26588 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:38.501103 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.523732 27077 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:38.492035 26919 consensus_queue.cc:1048] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:20:38.534333 26669 tablet_bootstrap.cc:654] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.539798 26670 tablet_bootstrap.cc:654] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.541958 26623 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:38.540524 27077 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.578410 26699 raft_consensus.cc:493] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.579155 26699 raft_consensus.cc:515] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.581405 26439 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:38.612695 26699 leader_election.cc:290] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:38.611716 27084 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:38.639725 27084 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.649853 27084 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:20:38.654515 26514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:37205, user_credentials={real_user=slave}} blocked reactor thread for 54096.6us
I20260709 12:20:38.629488 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:38.665947 27077 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:38.666685 26615 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:38.671084 27097 raft_consensus.cc:2764] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:20:38.673008 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.676872 26549 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.686470 26514 leader_election.cc:304] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e
I20260709 12:20:38.683027 27084 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:38.723984 26365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:37205, user_credentials={real_user=slave}} blocked reactor thread for 111909us
I20260709 12:20:38.731069 26469 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 1.
I20260709 12:20:38.732517 27084 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:38.735378 26669 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:38.735950 26669 ts_tablet_manager.cc:1403] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.348s	user 0.012s	sys 0.009s
I20260709 12:20:38.744410 26365 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:38.748961 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:38.749852 26474 raft_consensus.cc:2368] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:38.751148 27022 raft_consensus.cc:2804] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:38.751791 27022 raft_consensus.cc:697] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:38.752688 27022 consensus_queue.cc:237] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:38.765385 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:38.766580 26614 raft_consensus.cc:2393] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 2: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:38.778586 26365 leader_election.cc:400] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Vote denied by peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205) with higher term. Message: Invalid argument: T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:38.778934 26365 leader_election.cc:403] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:38.781826 26670 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:38.782332 26670 ts_tablet_manager.cc:1403] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.328s	user 0.018s	sys 0.033s
I20260709 12:20:38.781695 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:38.759512 26393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:38.786317 26615 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:38.793691 26669 raft_consensus.cc:359] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.794533 26669 raft_consensus.cc:385] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.793884 26298 catalog_manager.cc:5697] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.794822 26669 raft_consensus.cc:740] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.795823 26393 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:20:38.795576 26669 consensus_queue.cc:260] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
W20260709 12:20:38.821282 26345 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:38.826483 26699 raft_consensus.cc:2749] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:38.833719 27166 consensus_queue.cc:1048] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:38.845855 26669 ts_tablet_manager.cc:1434] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.110s	user 0.008s	sys 0.000s
I20260709 12:20:38.859100 26295 catalog_manager.cc:5697] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:38.866727 26670 raft_consensus.cc:359] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.803195 26615 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 3.
I20260709 12:20:38.803185 26393 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 3.
I20260709 12:20:38.868604 26670 raft_consensus.cc:385] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.872081 26670 raft_consensus.cc:740] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.872112 27126 raft_consensus.cc:2749] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205) with higher term. Message: Invalid argument: T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3.
I20260709 12:20:38.874253 26670 consensus_queue.cc:260] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.879020 26438 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:38.879846 26915 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:38.880286 26915 raft_consensus.cc:697] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:38.881145 26670 ts_tablet_manager.cc:1434] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.002s
I20260709 12:20:38.881136 26915 consensus_queue.cc:237] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:38.880263 27178 log.cc:927] Time spent T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:20:38.896535 26296 catalog_manager.cc:5697] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:38.899466 26670 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: Bootstrap starting.
I20260709 12:20:38.902738 26296 catalog_manager.cc:5697] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 1 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.906576 26669 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:38.915757 26394 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.917457 26670 tablet_bootstrap.cc:654] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.919940 26669 tablet_bootstrap.cc:654] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.930673 27137 consensus_queue.cc:1048] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:38.940155 27137 raft_consensus.cc:493] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.940635 27137 raft_consensus.cc:515] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.943086 26294 catalog_manager.cc:5697] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.951532 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:38.951849 27137 leader_election.cc:290] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
W20260709 12:20:38.952860 26589 leader_election.cc:343] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.954713 26393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:38.955473 26393 raft_consensus.cc:2468] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 0.
I20260709 12:20:38.956660 26586 leader_election.cc:304] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:38.957964 27137 raft_consensus.cc:2804] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:38.958405 27137 raft_consensus.cc:493] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.958817 27137 raft_consensus.cc:3060] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.966033 26669 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:38.966569 26669 ts_tablet_manager.cc:1403] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.003s
I20260709 12:20:38.975430 26670 tablet_bootstrap.cc:492] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: No bootstrap required, opened a new log
I20260709 12:20:38.976069 26670 ts_tablet_manager.cc:1403] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.004s
I20260709 12:20:38.979866 26669 raft_consensus.cc:359] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:38.980645 26669 raft_consensus.cc:385] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.980826 26669 raft_consensus.cc:740] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.981472 26669 consensus_queue.cc:260] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:38.982626 26670 raft_consensus.cc:359] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.983366 26670 raft_consensus.cc:385] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.983646 26670 raft_consensus.cc:740] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.984318 26670 consensus_queue.cc:260] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:38.988489 27137 raft_consensus.cc:515] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:38.991788 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:38.992133 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:38.992460 26394 raft_consensus.cc:3060] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:38.993501 26589 leader_election.cc:343] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.998111 26298 catalog_manager.cc:5697] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.983029 26669 ts_tablet_manager.cc:1434] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 12:20:39.008337 26669 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:39.018918 26669 tablet_bootstrap.cc:654] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.021548 26670 ts_tablet_manager.cc:1434] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.001s
I20260709 12:20:39.021262 26296 catalog_manager.cc:5697] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.022303 27137 leader_election.cc:290] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:39.026754 26394 raft_consensus.cc:2468] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:39.028107 26586 leader_election.cc:304] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:39.029304 27137 raft_consensus.cc:2804] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.029963 27137 raft_consensus.cc:697] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:39.031141 27137 consensus_queue.cc:237] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:39.038587 26669 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:39.039141 26669 ts_tablet_manager.cc:1403] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.011s
I20260709 12:20:39.047489 26669 raft_consensus.cc:359] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.048190 26669 raft_consensus.cc:385] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.075703 26669 raft_consensus.cc:740] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.076651 26669 consensus_queue.cc:260] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.079731 26669 ts_tablet_manager.cc:1434] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:20:39.083164 26669 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: Bootstrap starting.
I20260709 12:20:39.089849 26669 tablet_bootstrap.cc:654] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.091610 26298 catalog_manager.cc:5697] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 1 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.103711 26669 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: No bootstrap required, opened a new log
I20260709 12:20:39.104262 26669 ts_tablet_manager.cc:1403] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 12:20:39.106163 26669 raft_consensus.cc:359] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:39.106649 26669 raft_consensus.cc:385] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.106882 26669 raft_consensus.cc:740] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.107532 26669 consensus_queue.cc:260] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:39.109316 26669 ts_tablet_manager.cc:1434] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:39.143339 27137 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.143841 27137 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:39.146919 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:39.146976 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.147727 26394 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:39.147778 26468 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:39.149235 26588 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:39.149950 27164 raft_consensus.cc:2804] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:39.150074 27176 raft_consensus.cc:493] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.150367 27164 raft_consensus.cc:493] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.150731 27164 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:39.150637 27176 raft_consensus.cc:515] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:39.152952 27176 leader_election.cc:290] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:39.153736 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.153949 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:39.154449 26468 raft_consensus.cc:2468] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:39.154654 26615 raft_consensus.cc:2468] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 0.
I20260709 12:20:39.155673 26514 leader_election.cc:304] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:39.156672 27176 raft_consensus.cc:2804] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.157039 27176 raft_consensus.cc:493] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.157368 27176 raft_consensus.cc:3060] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.160641 27164 raft_consensus.cc:515] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:39.163406 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:39.153409 27137 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.164005 26394 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:39.167073 27164 leader_election.cc:290] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.168082 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc216f2a251463a9c406ab418b62360" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:39.168434 27176 raft_consensus.cc:515] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:39.168817 26468 raft_consensus.cc:3060] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:39.170418 27176 leader_election.cc:290] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:39.171025 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:39.171674 26470 raft_consensus.cc:3060] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.171622 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953f66c8075b448fa1ceb677489c5129" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:39.172564 26615 raft_consensus.cc:3060] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.172648 26394 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:39.174288 26586 leader_election.cc:304] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:39.177191 27137 raft_consensus.cc:2804] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:39.177686 27137 raft_consensus.cc:697] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:39.178534 27137 consensus_queue.cc:237] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:39.179294 26615 raft_consensus.cc:2468] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:39.180414 26513 leader_election.cc:304] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:39.180544 26470 raft_consensus.cc:2468] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:39.182533 27176 raft_consensus.cc:2804] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.183185 27176 raft_consensus.cc:697] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:39.184670 27176 consensus_queue.cc:237] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:39.193670 26298 catalog_manager.cc:5697] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 0 to 2, leader changed from <none> to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.195947 26468 raft_consensus.cc:2468] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:39.203743 26296 catalog_manager.cc:5697] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 0 to 1, leader changed from <none> to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 1 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.229977 27176 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.231207 27176 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.233804 27176 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.233913 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:39.234951 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.236166 26514 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:39.236778 27176 raft_consensus.cc:2749] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.263793 27126 consensus_queue.cc:1048] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.276072 27126 consensus_queue.cc:1048] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.327312 27174 raft_consensus.cc:493] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.327865 27174 raft_consensus.cc:515] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:39.331032 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:39.335774 27174 leader_election.cc:290] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:39.337127 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "095ead81e4364b36a759035a2def0683" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:39.340356 26394 raft_consensus.cc:2393] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 1: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:39.340445 26615 raft_consensus.cc:1275] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:39.341607 26512 leader_election.cc:304] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:39.342518 27121 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.342736 27174 raft_consensus.cc:3060] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.350934 27174 raft_consensus.cc:2749] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.355307 26394 raft_consensus.cc:1275] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:39.357019 26915 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.366111 27111 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:39.366631 27111 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.368624 27111 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:39.369333 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.369997 26468 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:39.371114 26365 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:39.371861 27111 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:39.371762 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:39.372542 27111 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bb83296e641848deaddf424cc59fda0e)
I20260709 12:20:39.372951 27111 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:39.373188 26545 raft_consensus.cc:2393] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 2: Already voted for candidate 297900273c7740af9938749b3ce3a80d in this term.
I20260709 12:20:39.382368 27111 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.388286 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:39.395913 27111 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:39.396306 26468 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:39.396916 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:39.397418 26365 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:39.397738 26545 raft_consensus.cc:2393] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fc524c91d3c249c0aeb08dae83b98e41 in current term 2: Already voted for candidate 297900273c7740af9938749b3ce3a80d in this term.
I20260709 12:20:39.398269 27126 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:39.398761 27126 raft_consensus.cc:697] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:39.399627 27126 consensus_queue.cc:237] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.410560 26295 catalog_manager.cc:5697] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.422032 27174 raft_consensus.cc:493] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.422675 27174 raft_consensus.cc:515] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.425086 27174 leader_election.cc:290] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.426139 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:39.426131 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.428705 26514 leader_election.cc:304] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:39.429406 27174 raft_consensus.cc:2749] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.463956 26547 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.543864 26547 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.600641 27137 consensus_queue.cc:1048] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.618206 27097 consensus_queue.cc:1048] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.620066 26468 raft_consensus.cc:1275] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:39.621622 27164 consensus_queue.cc:1048] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.645603 26394 raft_consensus.cc:1275] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:39.647083 27137 consensus_queue.cc:1048] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.754649 27174 consensus_queue.cc:1048] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.767694 27174 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.768150 27174 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.771001 27174 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.773150 26394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:39.774742 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:39.775429 26468 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:39.778039 26514 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:39.779515 27204 consensus_queue.cc:1048] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.781339 27204 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:39.781695 27204 raft_consensus.cc:493] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.782035 27204 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:39.790989 27204 raft_consensus.cc:515] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.795140 27204 leader_election.cc:290] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 3 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:39.797021 26393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:39.800068 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "235d0175b299499399baaa177b25de35" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:39.800660 26470 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:39.814199 26470 raft_consensus.cc:2468] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 3.
I20260709 12:20:39.815223 26514 leader_election.cc:304] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:39.815971 27204 raft_consensus.cc:2804] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:39.816416 27204 raft_consensus.cc:697] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 3 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:39.817260 27204 consensus_queue.cc:237] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.846544 26295 catalog_manager.cc:5697] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 2 to 3, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 3 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.891646 26470 raft_consensus.cc:1240] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Rejecting Update request from peer fc524c91d3c249c0aeb08dae83b98e41 for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:39.892961 27111 consensus_queue.cc:1059] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:39.894381 27126 raft_consensus.cc:3055] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:39.894766 27126 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:39.895596 27126 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:39.896840 27126 raft_consensus.cc:3060] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:40.162079 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
W20260709 12:20:40.164798 26586 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:40.172030 26586 consensus_peers.cc:597] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.197067 26512 consensus_peers.cc:597] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.202768 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:40.203869 17157 tablet_replica.cc:333] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.204730 17157 raft_consensus.cc:2243] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:20:40.203289 26438 consensus_peers.cc:597] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.205410 26438 consensus_peers.cc:597] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.205790 17157 raft_consensus.cc:2272] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.208348 26438 consensus_peers.cc:597] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.210649 17157 tablet_replica.cc:333] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.211642 17157 raft_consensus.cc:2243] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.212236 17157 raft_consensus.cc:2272] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.217819 26438 consensus_peers.cc:597] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.218456 17157 tablet_replica.cc:333] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.219511 17157 raft_consensus.cc:2243] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.220017 17157 pending_rounds.cc:70] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Trying to abort 2 pending ops.
I20260709 12:20:40.220286 17157 pending_rounds.cc:77] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305624122990440448 op_type: NO_OP noop_request { }
I20260709 12:20:40.220791 17157 raft_consensus.cc:2889] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:40.221158 17157 pending_rounds.cc:77] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305624111978467328 op_type: NO_OP noop_request { }
I20260709 12:20:40.221575 17157 raft_consensus.cc:2889] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:40.221993 17157 raft_consensus.cc:2272] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.224867 17157 tablet_replica.cc:333] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
W20260709 12:20:40.225149 26512 consensus_peers.cc:597] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.225843 17157 raft_consensus.cc:2243] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.226567 17157 raft_consensus.cc:2272] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.229164 17157 tablet_replica.cc:333] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
W20260709 12:20:40.230051 26586 consensus_peers.cc:597] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.230127 17157 raft_consensus.cc:2243] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.231210 17157 raft_consensus.cc:2272] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.233815 17157 tablet_replica.cc:333] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.234800 17157 raft_consensus.cc:2243] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.235358 17157 raft_consensus.cc:2272] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.237843 17157 tablet_replica.cc:333] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.238647 17157 raft_consensus.cc:2243] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.239133 17157 raft_consensus.cc:2272] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.241250 17157 tablet_replica.cc:333] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.242087 17157 raft_consensus.cc:2243] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.243110 17157 raft_consensus.cc:2272] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.245127 17157 tablet_replica.cc:333] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.245725 17157 raft_consensus.cc:2243] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.246598 17157 raft_consensus.cc:2272] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.248643 17157 tablet_replica.cc:333] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.249284 17157 raft_consensus.cc:2243] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.250041 17157 raft_consensus.cc:2272] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.251971 17157 tablet_replica.cc:333] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.252599 17157 raft_consensus.cc:2243] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.253374 17157 raft_consensus.cc:2272] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.255867 17157 tablet_replica.cc:333] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.256769 17157 raft_consensus.cc:2243] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.257292 17157 raft_consensus.cc:2272] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.257350 26438 consensus_peers.cc:597] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.259892 17157 tablet_replica.cc:333] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.260567 17157 raft_consensus.cc:2243] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.261329 17157 raft_consensus.cc:2272] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.263953 17157 tablet_replica.cc:333] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.264647 17157 raft_consensus.cc:2243] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.265116 17157 raft_consensus.cc:2272] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.267959 17157 tablet_replica.cc:333] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.268597 17157 raft_consensus.cc:2243] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.269063 17157 raft_consensus.cc:2272] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.270978 17157 tablet_replica.cc:333] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.271596 17157 raft_consensus.cc:2243] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.272312 17157 raft_consensus.cc:2272] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.274176 17157 tablet_replica.cc:333] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.274719 17157 raft_consensus.cc:2243] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.275254 17157 raft_consensus.cc:2272] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.277627 17157 tablet_replica.cc:333] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.278477 17157 raft_consensus.cc:2243] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.279029 17157 raft_consensus.cc:2272] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.281888 17157 tablet_replica.cc:333] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.282994 17157 raft_consensus.cc:2243] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.283699 17157 raft_consensus.cc:2272] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.286716 17157 tablet_replica.cc:333] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.287529 17157 raft_consensus.cc:2243] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:20:40.288157 26438 consensus_peers.cc:597] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.288553 17157 raft_consensus.cc:2272] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.290977 17157 tablet_replica.cc:333] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
W20260709 12:20:40.291127 26438 consensus_peers.cc:597] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.291687 17157 raft_consensus.cc:2243] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.292230 17157 raft_consensus.cc:2272] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.294409 17157 tablet_replica.cc:333] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.295038 17157 raft_consensus.cc:2243] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.295527 17157 raft_consensus.cc:2272] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.297559 17157 tablet_replica.cc:333] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.298133 17157 raft_consensus.cc:2243] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.298617 17157 raft_consensus.cc:2272] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.300500 17157 tablet_replica.cc:333] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.301092 17157 raft_consensus.cc:2243] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.301530 17157 raft_consensus.cc:2272] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.303896 17157 tablet_replica.cc:333] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.304507 17157 raft_consensus.cc:2243] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.305004 17157 raft_consensus.cc:2272] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.305719 26512 consensus_peers.cc:597] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.306991 17157 tablet_replica.cc:333] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.307654 17157 raft_consensus.cc:2243] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.308059 17157 raft_consensus.cc:2272] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.309824 17157 tablet_replica.cc:333] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.310397 17157 raft_consensus.cc:2243] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.310815 17157 raft_consensus.cc:2272] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.312598 17157 tablet_replica.cc:333] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.313143 17157 raft_consensus.cc:2243] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.313956 17157 raft_consensus.cc:2272] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.315825 17157 tablet_replica.cc:333] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.316341 17157 raft_consensus.cc:2243] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.316721 17157 raft_consensus.cc:2272] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.318470 17157 tablet_replica.cc:333] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.319092 17157 raft_consensus.cc:2243] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.319563 17157 raft_consensus.cc:2272] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.321844 17157 tablet_replica.cc:333] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.322398 17157 raft_consensus.cc:2243] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.322911 17157 raft_consensus.cc:2272] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.325610 17157 tablet_replica.cc:333] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.326224 17157 raft_consensus.cc:2243] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.327275 17157 raft_consensus.cc:2272] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.329154 17157 tablet_replica.cc:333] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.329748 17157 raft_consensus.cc:2243] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.330531 17157 raft_consensus.cc:2272] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.332415 17157 tablet_replica.cc:333] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.332952 17157 raft_consensus.cc:2243] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.333375 17157 raft_consensus.cc:2272] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.335150 17157 tablet_replica.cc:333] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.335736 17157 raft_consensus.cc:2243] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.336146 17157 raft_consensus.cc:2272] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.338002 17157 tablet_replica.cc:333] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.338649 17157 raft_consensus.cc:2243] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.339427 17157 raft_consensus.cc:2272] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.341293 17157 tablet_replica.cc:333] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.341837 17157 raft_consensus.cc:2243] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.342260 17157 raft_consensus.cc:2272] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.344286 17157 tablet_replica.cc:333] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.344834 17157 raft_consensus.cc:2243] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.345170 17157 raft_consensus.cc:2272] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.347117 17157 tablet_replica.cc:333] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.347823 17157 raft_consensus.cc:2243] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.348799 17157 raft_consensus.cc:2272] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.350991 26470 raft_consensus.cc:1275] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:40.351501 17157 tablet_replica.cc:333] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.352257 17157 raft_consensus.cc:2243] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.353225 17157 raft_consensus.cc:2272] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.352375 27204 consensus_queue.cc:1048] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 12:20:40.357199 26438 consensus_peers.cc:597] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.357542 17157 tablet_replica.cc:333] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.358834 17157 raft_consensus.cc:2243] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.359539 17157 raft_consensus.cc:2272] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.364629 17157 tablet_replica.cc:333] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.365446 17157 raft_consensus.cc:2243] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.366681 17157 raft_consensus.cc:2272] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.368966 17157 tablet_replica.cc:333] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.369963 17157 raft_consensus.cc:2243] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:40.371032 17157 raft_consensus.cc:2272] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.373378 26438 consensus_peers.cc:597] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.374547 17157 tablet_replica.cc:333] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.375761 17157 raft_consensus.cc:2243] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.376454 17157 raft_consensus.cc:2272] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:40.379144 17157 tablet_replica.cc:333] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: stopping tablet replica
I20260709 12:20:40.379772 17157 raft_consensus.cc:2243] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:40.380311 17157 raft_consensus.cc:2272] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:40.399721 26586 consensus_peers.cc:597] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.412315 26512 consensus_peers.cc:597] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.421937 26586 consensus_peers.cc:597] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.434338 26586 consensus_peers.cc:597] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.448509 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
W20260709 12:20:40.483723 26512 consensus_peers.cc:597] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.508252 26438 consensus_peers.cc:597] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.512190 26586 consensus_peers.cc:597] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.518451 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:40.524813 27216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:40.525202 27217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:40.528085 27219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:40.528373 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:40.533736 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:40.533943 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:40.534093 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599640534081 us; error 0 us; skew 500 ppm
I20260709 12:20:40.534627 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:40.537531 17157 webserver.cc:533] Webserver started at http://127.16.193.65:37971/ using document root <none> and password file <none>
I20260709 12:20:40.537966 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:40.538116 17157 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:20:40.541651 17157 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.000s	sys 0.004s
W20260709 12:20:40.547772 26586 consensus_peers.cc:597] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.550504 27224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:40.551504 17157 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
W20260709 12:20:40.551800 26586 consensus_peers.cc:597] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.551887 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "fc524c91d3c249c0aeb08dae83b98e41"
format_stamp: "Formatted at 2026-07-09 12:20:27 on dist-test-slave-n640"
I20260709 12:20:40.552501 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:20:40.577171 26438 consensus_peers.cc:597] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.595147 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:40.596583 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:40.598783 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:40.607004 27231 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:20:40.626138 26512 consensus_peers.cc:597] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.666615 26438 consensus_peers.cc:597] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.686460 26586 consensus_peers.cc:597] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:40.738185 26438 consensus_peers.cc:597] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:40.857800 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:20:40.858228 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.254s	user 0.012s	sys 0.000s
I20260709 12:20:40.858546 17157 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:20:40.863634 27231 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:40.907608 27231 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:40.908551 27231 tablet_bootstrap.cc:492] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:40.909178 27231 ts_tablet_manager.cc:1403] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.032s
I20260709 12:20:40.914004 27231 raft_consensus.cc:359] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:40.914667 27231 raft_consensus.cc:740] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.915378 27231 consensus_queue.cc:260] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:40.933620 27231 ts_tablet_manager.cc:1434] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 12:20:40.934419 27231 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:40.966260 27231 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:40.967155 27231 tablet_bootstrap.cc:492] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:40.967787 27231 ts_tablet_manager.cc:1403] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.034s	user 0.028s	sys 0.004s
I20260709 12:20:40.970124 27231 raft_consensus.cc:359] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:40.970780 27231 raft_consensus.cc:740] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.971534 27231 consensus_queue.cc:260] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:40.973464 27231 ts_tablet_manager.cc:1434] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:20:40.974248 27231 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:40.988489 27231 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:40.989333 27231 tablet_bootstrap.cc:492] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:40.989955 27231 ts_tablet_manager.cc:1403] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:20:40.992419 27231 raft_consensus.cc:359] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:40.993069 27231 raft_consensus.cc:740] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.993727 27231 consensus_queue.cc:260] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:40.995546 27231 ts_tablet_manager.cc:1434] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:20:40.996243 27231 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.007175 27231 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.007978 27231 tablet_bootstrap.cc:492] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.008539 27231 ts_tablet_manager.cc:1403] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:20:41.010913 27231 raft_consensus.cc:359] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.011564 27231 raft_consensus.cc:740] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.012341 27231 consensus_queue.cc:260] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.014256 27231 ts_tablet_manager.cc:1434] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:41.014953 27231 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.068421 27231 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.069319 27231 tablet_bootstrap.cc:492] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.069964 27231 ts_tablet_manager.cc:1403] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.055s	user 0.027s	sys 0.015s
I20260709 12:20:41.072372 27231 raft_consensus.cc:359] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.073027 27231 raft_consensus.cc:740] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.073647 27231 consensus_queue.cc:260] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.075835 27231 ts_tablet_manager.cc:1434] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:41.076609 27231 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.103317 27231 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.104228 27231 tablet_bootstrap.cc:492] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.104848 27231 ts_tablet_manager.cc:1403] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.008s
I20260709 12:20:41.107065 27231 raft_consensus.cc:359] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.107758 27231 raft_consensus.cc:740] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.108404 27231 consensus_queue.cc:260] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.110496 27231 ts_tablet_manager.cc:1434] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:41.111205 27231 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.124251 27231 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.125106 27231 tablet_bootstrap.cc:492] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.125671 27231 ts_tablet_manager.cc:1403] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 12:20:41.128082 27231 raft_consensus.cc:359] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.128727 27231 raft_consensus.cc:740] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.129472 27231 consensus_queue.cc:260] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.131524 27231 ts_tablet_manager.cc:1434] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:20:41.132272 27231 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.145910 27231 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.146860 27231 tablet_bootstrap.cc:492] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.147470 27231 ts_tablet_manager.cc:1403] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 12:20:41.149650 27231 raft_consensus.cc:359] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.150318 27231 raft_consensus.cc:740] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.150960 27231 consensus_queue.cc:260] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.152689 17157 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:20:41.152989 27231 ts_tablet_manager.cc:1434] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:20:41.153915 27231 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.153046 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.295s	user 0.246s	sys 0.004s
I20260709 12:20:41.170955 27231 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.171900 27231 tablet_bootstrap.cc:492] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.172510 27231 ts_tablet_manager.cc:1403] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:20:41.174166 27239 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.174742 27239 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.175184 27231 raft_consensus.cc:359] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.176191 27231 raft_consensus.cc:740] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.176973 27239 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.177273 27231 consensus_queue.cc:260] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.180370 27231 ts_tablet_manager.cc:1434] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:41.181360 27231 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:41.182823 26586 leader_election.cc:336] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111)
I20260709 12:20:41.185141 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.187341 26588 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.188606 27239 raft_consensus.cc:2749] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.197520 27231 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.198331 27231 tablet_bootstrap.cc:492] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.198905 27231 ts_tablet_manager.cc:1403] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:20:41.201193 27231 raft_consensus.cc:359] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.201741 27231 raft_consensus.cc:740] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.202391 27231 consensus_queue.cc:260] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.204113 27231 ts_tablet_manager.cc:1434] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:20:41.204752 27231 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.225272 27231 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.226287 27231 tablet_bootstrap.cc:492] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.226943 27231 ts_tablet_manager.cc:1403] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.007s
I20260709 12:20:41.228960 27231 raft_consensus.cc:359] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.229781 27231 raft_consensus.cc:740] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.230441 27231 consensus_queue.cc:260] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.232568 27231 ts_tablet_manager.cc:1434] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.009s	sys 0.000s
I20260709 12:20:41.233343 27231 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.249934 27231 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.250870 27231 tablet_bootstrap.cc:492] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.251569 27231 ts_tablet_manager.cc:1403] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:20:41.253448 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:41581
I20260709 12:20:41.253765 27302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:41581 every 8 connection(s)
I20260709 12:20:41.254158 27231 raft_consensus.cc:359] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.255040 27231 raft_consensus.cc:740] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.255712 27231 consensus_queue.cc:260] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.257973 27231 ts_tablet_manager.cc:1434] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:41.258684 27231 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.262367 27174 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.264130 27174 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.266014 27174 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.266569 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.306818 27137 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.307379 27137 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.310199 27142 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.312152 26915 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.310771 27142 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.312662 26915 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.314540 26915 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.314739 27142 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.316071 27137 leader_election.cc:290] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.316542 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.316869 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.317490 26623 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.318737 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.319049 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.320600 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.321811 26439 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:41.322813 26915 raft_consensus.cc:2804] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.323213 26915 raft_consensus.cc:493] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.323653 26915 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.333076 26438 leader_election.cc:343] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:41.334059 26438 leader_election.cc:343] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.334440 26438 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.336408 27142 raft_consensus.cc:2749] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.338714 27231 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.339767 27231 tablet_bootstrap.cc:492] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.340515 27231 ts_tablet_manager.cc:1403] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.082s	user 0.021s	sys 0.021s
I20260709 12:20:41.341879 27304 heartbeater.cc:344] Connected to a master server at 127.16.193.126:33125
I20260709 12:20:41.342357 27304 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:41.343344 27304 heartbeater.cc:507] Master 127.16.193.126:33125 requested a full tablet report, sending...
I20260709 12:20:41.343420 27231 raft_consensus.cc:359] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.344377 27231 raft_consensus.cc:740] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.345093 27231 consensus_queue.cc:260] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.346680 27231 ts_tablet_manager.cc:1434] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:20:41.347584 27231 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.352821 26915 raft_consensus.cc:515] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.354856 26915 leader_election.cc:290] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.359390 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
W20260709 12:20:41.361063 26438 leader_election.cc:343] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.364274 27231 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260709 12:20:41.364303 26296 ts_manager.cc:194] Re-registered known tserver with Master: fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.365602 27231 tablet_bootstrap.cc:492] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.366286 27231 ts_tablet_manager.cc:1403] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:20:41.368678 27231 raft_consensus.cc:359] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.369558 27231 raft_consensus.cc:740] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.370223 27231 consensus_queue.cc:260] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.364367 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.371793 27231 ts_tablet_manager.cc:1434] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
W20260709 12:20:41.372709 26512 leader_election.cc:343] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.373389 26512 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.374796 27176 raft_consensus.cc:2749] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.374917 27276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.375528 27176 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.375952 27176 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:41.376300 26586 leader_election.cc:343] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.376757 26586 leader_election.cc:304] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.372716 27231 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.377777 27137 raft_consensus.cc:2749] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.378098 27176 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.381456 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.382679 26512 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.386245 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063cdd050c3d40fdab471dec902e20e7" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:41.386829 26623 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.392037 27313 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.394672 27313 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.397784 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.399259 26438 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.400274 27313 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.402531 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.403147 26547 raft_consensus.cc:2468] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.408931 26441 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.410295 27311 raft_consensus.cc:2804] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.410336 27137 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.410828 27311 raft_consensus.cc:493] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.411288 27311 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.411173 27137 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.414839 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.415961 27274 raft_consensus.cc:2468] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.417363 26586 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:41.414834 27137 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.419466 27239 raft_consensus.cc:2804] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.419764 27311 raft_consensus.cc:515] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.420042 27239 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.420733 27239 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.421926 27311 leader_election.cc:290] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.425935 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.426134 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.426658 27142 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.427193 27142 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:41.428499 26438 leader_election.cc:343] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.429163 27142 leader_election.cc:290] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.430769 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.433167 27204 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.433862 27204 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:41.433173 26438 leader_election.cc:343] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.435369 27239 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.437510 27204 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.438284 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.438481 27276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.438862 27277 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.440380 26512 leader_election.cc:343] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.441355 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.441884 26547 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.443567 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.446722 26513 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.447485 27231 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.448444 27231 tablet_bootstrap.cc:492] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.449092 27231 ts_tablet_manager.cc:1403] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.077s	user 0.031s	sys 0.030s
I20260709 12:20:41.449126 27166 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.449798 27166 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.452206 27166 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.455592 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.451967 27231 raft_consensus.cc:359] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.456439 26470 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.456550 27231 raft_consensus.cc:740] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.457275 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a289521cf13242cf80f6cf90c53b8ab4" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.457867 27231 consensus_queue.cc:260] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.458120 26468 raft_consensus.cc:2393] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 2: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:41.458084 27204 raft_consensus.cc:2749] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.459190 26588 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:41.459909 26296 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47970
I20260709 12:20:41.460134 27166 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.460289 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.460505 27231 ts_tablet_manager.cc:1434] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:20:41.460681 27166 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.461140 27166 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.461599 26586 leader_election.cc:343] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.462375 27231 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.464721 27304 heartbeater.cc:499] Master 127.16.193.126:33125 was elected leader, sending a full tablet report...
I20260709 12:20:41.466574 26915 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.466922 27277 raft_consensus.cc:2468] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.467065 26915 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.467839 27239 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.468190 26586 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:41.468969 27232 raft_consensus.cc:2804] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.469475 27232 raft_consensus.cc:697] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:41.470014 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:41.470623 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.471951 26438 leader_election.cc:343] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.472833 27204 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.470371 27232 consensus_queue.cc:237] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.473403 27204 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.476312 27273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.477029 27176 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.477550 27176 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:41.477815 26512 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.479801 27176 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.480252 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.481885 26512 leader_election.cc:343] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.482275 27176 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.482431 27204 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.482826 27176 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.485505 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.486127 27176 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.488001 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.488907 26614 raft_consensus.cc:2468] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
W20260709 12:20:41.493233 26512 leader_election.cc:343] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.493990 26514 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.495548 27137 raft_consensus.cc:493] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.496061 27137 raft_consensus.cc:515] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.497069 27312 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.498133 27137 leader_election.cc:290] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.499082 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.499836 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.497565 27312 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
W20260709 12:20:41.501101 26586 leader_election.cc:343] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.501551 26586 leader_election.cc:304] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.503196 27273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.503481 27239 raft_consensus.cc:2749] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:41.504680 26438 leader_election.cc:343] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.508095 27166 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.509501 27137 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.510147 27204 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.511538 27137 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.511643 27204 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.513630 27137 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.513895 27204 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.514638 27231 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.515100 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.515836 26470 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.516310 27231 tablet_bootstrap.cc:492] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:41.516722 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.510468 26623 raft_consensus.cc:2468] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:41.517134 26588 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:41.518591 26547 raft_consensus.cc:2468] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:41.519707 27176 raft_consensus.cc:2749] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:41.519938 26512 leader_election.cc:343] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.520552 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.521024 26441 leader_election.cc:304] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.521906 26513 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.522109 27311 raft_consensus.cc:2804] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.522616 27311 raft_consensus.cc:697] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:41.519761 27137 raft_consensus.cc:2804] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.522678 27176 raft_consensus.cc:2749] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.523255 27137 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.523730 27231 ts_tablet_manager.cc:1403] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.006s
I20260709 12:20:41.523926 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.524576 26545 raft_consensus.cc:2468] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.519784 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.525695 26441 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.523695 27137 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.526372 26586 leader_election.cc:343] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.526644 27311 consensus_queue.cc:237] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.527021 27231 raft_consensus.cc:359] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.527878 27231 raft_consensus.cc:740] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.528632 27231 consensus_queue.cc:260] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.530244 27313 raft_consensus.cc:2804] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.530634 27313 raft_consensus.cc:493] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.531004 27313 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.537132 27231 ts_tablet_manager.cc:1434] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.001s
I20260709 12:20:41.540081 27313 raft_consensus.cc:515] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.537719 27312 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.542778 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.543200 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
W20260709 12:20:41.544634 26438 leader_election.cc:343] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.545013 27142 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.520016 26915 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.545532 27142 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.549183 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.549678 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
W20260709 12:20:41.550505 26438 leader_election.cc:343] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.551472 26441 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.552548 27177 raft_consensus.cc:2749] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.540547 27231 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.552558 27142 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.556169 27176 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.556692 27176 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.557684 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.558251 26547 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.558825 27176 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.560354 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.560727 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.560354 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:41.561941 26470 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.562319 26512 leader_election.cc:343] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.564213 26298 catalog_manager.cc:5697] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.567417 26439 leader_election.cc:304] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.568013 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
W20260709 12:20:41.570369 26586 leader_election.cc:343] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.571256 27142 raft_consensus.cc:2804] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.571624 27142 raft_consensus.cc:493] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.572016 27142 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.580008 27313 leader_election.cc:290] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.580358 27142 raft_consensus.cc:515] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.580489 26470 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.553594 27166 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.582278 27137 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.582934 26513 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.584535 26439 leader_election.cc:304] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.593086 27321 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.598403 27204 raft_consensus.cc:2749] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.601089 26915 raft_consensus.cc:2804] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.601703 26915 raft_consensus.cc:697] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:41.603508 27137 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.604451 26614 raft_consensus.cc:2393] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 2: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:41.602526 26915 consensus_queue.cc:237] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.598621 27321 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.609809 26588 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.606495 27166 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.617250 27166 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.617658 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.618294 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.616478 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.619377 27166 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.619879 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429d15dc784e4494ad111c92f4812cc6" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.620332 27319 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.620653 26468 raft_consensus.cc:2393] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 2: Already voted for candidate bb83296e641848deaddf424cc59fda0e in this term.
I20260709 12:20:41.620885 27319 raft_consensus.cc:697] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:41.621191 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.621994 26586 leader_election.cc:343] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.622975 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
W20260709 12:20:41.623410 26586 leader_election.cc:343] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.623577 26468 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.625156 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.625350 27277 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.626744 26586 leader_election.cc:343] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:41.627300 26438 leader_election.cc:343] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.628933 27166 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.629295 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.629469 27166 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.629972 26545 raft_consensus.cc:2468] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.630383 27319 consensus_queue.cc:237] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.631392 27142 leader_election.cc:290] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.631575 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.631690 26589 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.632189 26545 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.632766 27232 raft_consensus.cc:2804] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.633205 27232 raft_consensus.cc:493] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.633667 27232 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.635010 27166 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.635910 26468 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.637431 26588 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.638182 27324 raft_consensus.cc:2804] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.614110 26547 raft_consensus.cc:2468] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:41.638697 27324 raft_consensus.cc:697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:41.639144 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.639382 26441 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.639842 27274 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.640519 27142 raft_consensus.cc:2804] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.641055 27142 raft_consensus.cc:697] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:41.641206 26586 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:41.642239 27320 raft_consensus.cc:2804] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.642025 27142 consensus_queue.cc:237] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.642638 27320 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.643003 27320 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.650864 27137 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.651619 27137 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.651924 27320 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.653801 27137 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.629393 27321 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.655472 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.655565 26514 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.656675 27273 pending_rounds.cc:85] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Aborting all ops after (but not including) 1
I20260709 12:20:41.656872 26514 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.657145 27273 pending_rounds.cc:107] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:20:41.657610 27273 raft_consensus.cc:2889] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:41.658044 26514 leader_election.cc:304] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.659263 26439 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.660436 26589 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.662563 26547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.662868 27137 raft_consensus.cc:2804] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.664604 27137 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.665468 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff188538952450b970105de07be9a85" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:41.665596 27137 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.666318 26623 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.665429 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.653965 27320 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.677241 26623 raft_consensus.cc:2468] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:41.681949 26439 leader_election.cc:304] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.681428 26295 catalog_manager.cc:5697] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.658742 27312 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.657634 27204 raft_consensus.cc:2749] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.667742 27176 raft_consensus.cc:2749] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.670091 27313 raft_consensus.cc:2749] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.669488 27174 raft_consensus.cc:2749] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:41.693742 26586 leader_election.cc:343] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.684417 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.698796 27137 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.701759 26589 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
W20260709 12:20:41.709359 26514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:37205, user_credentials={real_user=slave}} blocked reactor thread for 49946.9us
I20260709 12:20:41.709607 27330 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.710151 27330 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.713173 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.713793 26623 raft_consensus.cc:2468] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:41.715298 27137 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.639595 27324 consensus_queue.cc:237] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.692544 27177 raft_consensus.cc:2804] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.720052 26513 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:41.717293 27176 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.693023 27312 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.726787 27174 raft_consensus.cc:2804] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.742585 26915 raft_consensus.cc:493] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.728183 27313 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.729604 27276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.731220 27177 raft_consensus.cc:697] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:41.743111 26915 raft_consensus.cc:515] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.739928 27166 raft_consensus.cc:2749] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.737627 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.736161 27330 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.760064 26915 leader_election.cc:290] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.760558 27177 consensus_queue.cc:237] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.780795 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.748287 27313 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.783869 27313 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.766805 27312 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.744302 27174 raft_consensus.cc:493] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.789661 27174 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.791582 27274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.772778 27272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81dffbd4eb274c0595db3619e8eb2058" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.800457 27272 raft_consensus.cc:2393] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb83296e641848deaddf424cc59fda0e in current term 2: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:41.773685 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.751940 27276 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.812399 26438 leader_election.cc:343] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.816444 27231 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:41.734557 27176 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.819980 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.821930 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.823482 27313 raft_consensus.cc:493] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.824048 26546 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.824430 27176 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:41.824143 27313 raft_consensus.cc:515] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.825865 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.826480 27313 leader_election.cc:290] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.826634 26545 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.758723 27232 raft_consensus.cc:515] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:41.827888 26512 leader_election.cc:343] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.830188 27232 leader_election.cc:290] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.830382 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.736488 27334 raft_consensus.cc:493] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.831341 27334 raft_consensus.cc:515] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.832336 26441 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.832909 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.834085 26441 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:41.834378 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.834988 26915 raft_consensus.cc:2804] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.835179 26615 raft_consensus.cc:2393] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 297900273c7740af9938749b3ce3a80d in current term 2: Already voted for candidate 39930cd9790d4943bdec6f9df4ce2641 in this term.
I20260709 12:20:41.835407 26915 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
W20260709 12:20:41.835588 26512 leader_election.cc:343] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.835803 26915 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.836695 27312 raft_consensus.cc:2804] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.837129 27312 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.837531 27312 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.842672 26915 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
W20260709 12:20:41.851473 26438 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.854171 27268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.854480 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.855085 26545 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.834776 26586 leader_election.cc:343] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.846247 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
W20260709 12:20:41.857990 26586 leader_election.cc:343] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.860105 27174 raft_consensus.cc:515] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.866422 26545 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:41.876260 26915 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:41.877625 27334 leader_election.cc:290] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.878273 27231 tablet_bootstrap.cc:492] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
W20260709 12:20:41.878793 26438 leader_election.cc:343] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.886185 27231 ts_tablet_manager.cc:1403] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.346s	user 0.018s	sys 0.017s
W20260709 12:20:41.886633 26438 leader_election.cc:343] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.886971 27174 leader_election.cc:290] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.887728 26441 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
W20260709 12:20:41.888824 26589 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 187130us
I20260709 12:20:41.889814 26615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:41.889559 27231 raft_consensus.cc:359] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.890522 26615 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.890605 27231 raft_consensus.cc:740] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.891513 27336 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.891999 27342 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.892040 27336 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.892702 27342 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:41.893468 27231 consensus_queue.cc:260] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.894374 27336 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.879084 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86ba667160b44ddf81a0d48eedb7bdae" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.892436 27272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "237bd2b722334bda9aeb29035546ca4c" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:41.895941 27268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.896335 26512 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 297900273c7740af9938749b3ce3a80d; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.896606 27268 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
W20260709 12:20:41.897480 26512 leader_election.cc:343] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.898320 27311 raft_consensus.cc:2804] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.898845 27311 raft_consensus.cc:697] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:41.899685 27311 consensus_queue.cc:237] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.901520 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00aa72f3c874a77b2d226ecbebbe834" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.902153 26546 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.902577 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:41.902954 26548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.903280 26545 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.903613 26548 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.905306 27334 raft_consensus.cc:3060] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.909938 27231 ts_tablet_manager.cc:1434] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:20:41.911052 27231 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:41.913398 27342 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:41.921173 27271 raft_consensus.cc:1275] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:41.922667 27268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41" is_pre_election: true
I20260709 12:20:41.826520 26439 leader_election.cc:304] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb83296e641848deaddf424cc59fda0e; no voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41
W20260709 12:20:41.928280 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 101826us
I20260709 12:20:41.929509 27335 raft_consensus.cc:3060] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.932670 26623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641" is_pre_election: true
I20260709 12:20:41.933312 26623 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 1.
I20260709 12:20:41.955258 26468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e" is_pre_election: true
I20260709 12:20:41.955942 26468 raft_consensus.cc:2468] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 1.
I20260709 12:20:41.959175 27335 raft_consensus.cc:2749] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.970789 26545 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.972259 26586 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
W20260709 12:20:41.973616 26586 leader_election.cc:343] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.975081 26439 leader_election.cc:304] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.970399 27334 raft_consensus.cc:2749] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:41.977933 27311 raft_consensus.cc:2804] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.979148 27311 raft_consensus.cc:493] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.883036 27276 raft_consensus.cc:2468] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.979540 27311 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:41.976930 27344 log.cc:927] Time spent T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:20:41.980655 26546 raft_consensus.cc:2468] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:41.977057 27232 raft_consensus.cc:2804] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.982913 27232 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:41.983340 27232 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.987775 27311 raft_consensus.cc:515] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:41.976770 27336 consensus_queue.cc:1048] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:20:41.990782 27312 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:41.978475 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d" is_pre_election: true
I20260709 12:20:41.996534 26545 raft_consensus.cc:2468] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 1.
I20260709 12:20:41.998497 26586 leader_election.cc:304] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 297900273c7740af9938749b3ce3a80d
I20260709 12:20:41.999732 27273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:42.001118 27336 raft_consensus.cc:2804] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.001637 27336 raft_consensus.cc:697] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:42.002278 26589 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:41.992249 27311 leader_election.cc:290] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:42.002463 27336 consensus_queue.cc:237] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.003834 27319 raft_consensus.cc:2804] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.004299 27319 raft_consensus.cc:697] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:42.005131 27319 consensus_queue.cc:237] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:42.008373 26438 leader_election.cc:343] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.009471 26514 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.010619 27176 raft_consensus.cc:2804] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:42.010990 27176 raft_consensus.cc:493] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:42.011409 27176 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.017251 26589 leader_election.cc:304] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.017171 27232 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.037142 27312 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:42.038154 27232 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:42.028343 26295 catalog_manager.cc:5697] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.041607 26295 catalog_manager.cc:5697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.047499 27276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
W20260709 12:20:42.049650 26438 consensus_peers.cc:597] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:42.050323 26438 leader_election.cc:343] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.050983 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:42.051316 27273 raft_consensus.cc:3060] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.051635 26546 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.053129 26548 raft_consensus.cc:1275] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.058669 26546 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:42.030471 26296 catalog_manager.cc:5697] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.034515 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "899048d400bc4540a891c445bcd9bd25" candidate_uuid: "bb83296e641848deaddf424cc59fda0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:42.059783 26614 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.059832 27273 raft_consensus.cc:1275] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.061579 27341 consensus_queue.cc:1048] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.035728 26615 raft_consensus.cc:2468] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:42.039803 27176 raft_consensus.cc:515] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.065912 26513 leader_election.cc:304] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.067207 26296 catalog_manager.cc:5697] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.068285 26441 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.070204 27176 leader_election.cc:290] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:42.069393 26296 catalog_manager.cc:5697] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.070840 27271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:42.071511 27271 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.073041 27337 raft_consensus.cc:2804] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.073604 27337 raft_consensus.cc:697] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:42.070873 27177 consensus_queue.cc:1048] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:20:42.074581 27337 consensus_queue.cc:237] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.080660 27335 raft_consensus.cc:2804] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.081262 27335 raft_consensus.cc:697] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:42.082144 27335 consensus_queue.cc:237] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.088037 27271 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:42.090256 26614 raft_consensus.cc:2468] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb83296e641848deaddf424cc59fda0e in term 2.
I20260709 12:20:42.096654 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:42.106894 26470 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.112241 27232 raft_consensus.cc:2804] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.112964 27232 raft_consensus.cc:697] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
W20260709 12:20:42.112882 26589 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 95703.7us
I20260709 12:20:42.115494 26589 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.115592 27232 consensus_queue.cc:237] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.117432 26586 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:42.125797 27239 raft_consensus.cc:2804] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:42.126135 27137 raft_consensus.cc:2804] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.126323 27239 raft_consensus.cc:493] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:42.126729 27137 raft_consensus.cc:697] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:42.126814 27239 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.128293 27137 consensus_queue.cc:237] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.135141 26439 leader_election.cc:304] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.098170 26296 catalog_manager.cc:5697] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.100950 27273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a97fd66be7ff4a588ce7be8aa1a5b5be" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
I20260709 12:20:42.143821 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:42.148039 26545 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.154330 26615 raft_consensus.cc:1275] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.155351 26470 raft_consensus.cc:2468] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:42.156265 27341 raft_consensus.cc:2804] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:20:42.157090 26512 leader_election.cc:343] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.159812 27341 raft_consensus.cc:697] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Leader. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:42.160213 27239 raft_consensus.cc:515] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.161058 27341 consensus_queue.cc:237] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.162367 27239 leader_election.cc:290] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:42.163975 27276 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
W20260709 12:20:42.165690 26586 leader_election.cc:343] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.166673 26514 leader_election.cc:304] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.166684 26915 consensus_queue.cc:1048] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:42.164533 26295 catalog_manager.cc:5697] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.174561 27176 raft_consensus.cc:2804] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.186242 27176 raft_consensus.cc:697] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:42.187533 27176 consensus_queue.cc:237] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.188740 26543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d9fdb0a26d4b43a1940f3ca15c530f" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:42.189533 26543 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.198670 26545 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:42.199867 26548 raft_consensus.cc:1275] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.202425 27312 consensus_queue.cc:1048] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.205214 26543 raft_consensus.cc:2468] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:42.206338 26468 raft_consensus.cc:1275] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:42.202947 26586 consensus_peers.cc:597] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:42.208155 26586 consensus_peers.cc:597] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.208689 27321 consensus_queue.cc:1048] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.210438 26589 leader_election.cc:304] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:42.211864 27336 raft_consensus.cc:2804] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.212355 27336 raft_consensus.cc:697] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:42.221999 26470 raft_consensus.cc:1275] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:42.229503 26438 consensus_peers.cc:597] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.232995 27231 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.232074 27336 consensus_queue.cc:237] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.236665 27231 tablet_bootstrap.cc:492] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.237025 27324 consensus_queue.cc:1048] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.237599 27231 ts_tablet_manager.cc:1403] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.327s	user 0.032s	sys 0.075s
I20260709 12:20:42.256494 27231 raft_consensus.cc:359] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.285683 27231 raft_consensus.cc:740] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.295397 27231 consensus_queue.cc:260] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.308521 27231 ts_tablet_manager.cc:1434] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.003s
I20260709 12:20:42.328332 26546 raft_consensus.cc:1275] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.330096 27335 consensus_queue.cc:1048] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:42.332000 27231 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.352030 27276 raft_consensus.cc:3060] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.372838 26470 raft_consensus.cc:1275] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.374433 27232 consensus_queue.cc:1048] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:42.382844 26438 consensus_peers.cc:597] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.386090 26614 raft_consensus.cc:1275] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.386839 27276 raft_consensus.cc:1275] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.362198 26298 catalog_manager.cc:5697] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.387883 27341 consensus_queue.cc:1048] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.388723 27333 consensus_queue.cc:1048] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.388804 26295 catalog_manager.cc:5697] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.391515 26295 catalog_manager.cc:5697] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to bb83296e641848deaddf424cc59fda0e (127.16.193.66). New cstate: current_term: 2 leader_uuid: "bb83296e641848deaddf424cc59fda0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.388890 26298 catalog_manager.cc:5697] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.394817 26298 catalog_manager.cc:5697] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.469993 26296 catalog_manager.cc:5697] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.480643 26545 raft_consensus.cc:1275] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.482151 27231 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.482421 27312 consensus_queue.cc:1048] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.483176 27231 tablet_bootstrap.cc:492] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.483952 27231 ts_tablet_manager.cc:1403] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.152s	user 0.038s	sys 0.029s
I20260709 12:20:42.484262 26614 raft_consensus.cc:1275] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.486028 27337 consensus_queue.cc:1048] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.486403 27231 raft_consensus.cc:359] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.487498 27231 raft_consensus.cc:740] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.488327 27231 consensus_queue.cc:260] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.492548 27231 ts_tablet_manager.cc:1434] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:42.493408 27231 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.495774 27276 raft_consensus.cc:1275] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.497671 27324 consensus_queue.cc:1048] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.503669 26543 raft_consensus.cc:1275] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.505394 27321 consensus_queue.cc:1048] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.531586 27231 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.532817 27231 tablet_bootstrap.cc:492] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.533701 27231 ts_tablet_manager.cc:1403] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.011s
I20260709 12:20:42.532714 26546 raft_consensus.cc:1275] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.536355 27231 raft_consensus.cc:359] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.536520 27232 consensus_queue.cc:1048] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:42.539436 27231 raft_consensus.cc:740] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.540362 27231 consensus_queue.cc:260] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.542977 27231 ts_tablet_manager.cc:1434] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
W20260709 12:20:42.560220 26438 consensus_peers.cc:597] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.564268 27231 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.569337 26470 raft_consensus.cc:1275] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.571189 27337 consensus_queue.cc:1048] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:42.599435 26512 consensus_peers.cc:597] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.616196 27271 raft_consensus.cc:1275] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.620702 27336 consensus_queue.cc:1048] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:42.626303 26438 consensus_peers.cc:597] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.616791 27231 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:20:42.618892 26512 consensus_peers.cc:597] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.628281 27231 tablet_bootstrap.cc:492] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.631498 27231 ts_tablet_manager.cc:1403] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.067s	user 0.021s	sys 0.022s
I20260709 12:20:42.634135 27231 raft_consensus.cc:359] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.634858 27231 raft_consensus.cc:740] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.635589 27231 consensus_queue.cc:260] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.637447 27231 ts_tablet_manager.cc:1434] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:42.638162 27231 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.654044 26294 catalog_manager.cc:5697] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:42.655743 26586 consensus_peers.cc:597] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.656303 26614 raft_consensus.cc:1275] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer bb83296e641848deaddf424cc59fda0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.657047 26294 catalog_manager.cc:5697] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.664661 27341 consensus_queue.cc:1048] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.670800 26543 raft_consensus.cc:1275] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.671123 27231 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.672149 27231 tablet_bootstrap.cc:492] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.672946 27231 ts_tablet_manager.cc:1403] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.008s
I20260709 12:20:42.672708 27321 consensus_queue.cc:1048] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.675765 27231 raft_consensus.cc:359] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.676472 27231 raft_consensus.cc:740] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.677233 27231 consensus_queue.cc:260] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.678828 26545 raft_consensus.cc:1275] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.679704 27231 ts_tablet_manager.cc:1434] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:42.680604 27231 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.683380 27232 consensus_queue.cc:1048] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.703817 27231 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.704859 27231 tablet_bootstrap.cc:492] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.705533 27231 ts_tablet_manager.cc:1403] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:20:42.708033 27231 raft_consensus.cc:359] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.708819 27231 raft_consensus.cc:740] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.709662 27231 consensus_queue.cc:260] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:42.713066 26438 consensus_peers.cc:597] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:42.714612 27231 ts_tablet_manager.cc:1434] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:20:42.715571 27231 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:42.717423 26586 consensus_peers.cc:597] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.731031 27231 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.732064 27231 tablet_bootstrap.cc:492] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.732829 27231 ts_tablet_manager.cc:1403] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 12:20:42.735340 27231 raft_consensus.cc:359] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.736047 27231 raft_consensus.cc:740] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.736797 27231 consensus_queue.cc:260] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.738731 27231 ts_tablet_manager.cc:1434] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:20:42.745113 27231 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:42.745914 26586 consensus_peers.cc:597] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:42.761672 27231 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.762758 27231 tablet_bootstrap.cc:492] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.763470 27231 ts_tablet_manager.cc:1403] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.002s
I20260709 12:20:42.766050 27231 raft_consensus.cc:359] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.766805 27231 raft_consensus.cc:740] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.767712 27231 consensus_queue.cc:260] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.769569 27231 ts_tablet_manager.cc:1434] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:20:42.770284 27231 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.782956 27231 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.783717 27231 tablet_bootstrap.cc:492] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.784348 27231 ts_tablet_manager.cc:1403] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
W20260709 12:20:42.784266 26512 consensus_peers.cc:597] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:42.786880 27231 raft_consensus.cc:359] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.787622 27231 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.788334 27231 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.790639 27231 ts_tablet_manager.cc:1434] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:20:42.792950 27231 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.806677 27231 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.807493 27231 tablet_bootstrap.cc:492] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.807997 27231 ts_tablet_manager.cc:1403] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:20:42.809769 27231 raft_consensus.cc:359] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.810487 27231 raft_consensus.cc:740] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.811028 27231 consensus_queue.cc:260] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:42.811872 26586 consensus_peers.cc:597] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.812943 27231 ts_tablet_manager.cc:1434] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.001s
I20260709 12:20:42.814143 27231 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.827883 27231 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.828542 27231 tablet_bootstrap.cc:492] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.829128 27231 ts_tablet_manager.cc:1403] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:20:42.831305 27231 raft_consensus.cc:359] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.831779 27231 raft_consensus.cc:740] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.832311 27231 consensus_queue.cc:260] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.834204 27231 ts_tablet_manager.cc:1434] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:42.836764 27231 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.840055 26545 raft_consensus.cc:1275] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:42.841980 27137 consensus_queue.cc:1048] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:42.869112 27231 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.870144 27231 tablet_bootstrap.cc:492] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.870759 27231 ts_tablet_manager.cc:1403] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.016s
I20260709 12:20:42.873327 27231 raft_consensus.cc:359] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.873855 27231 raft_consensus.cc:740] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.874522 27231 consensus_queue.cc:260] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.876309 27231 ts_tablet_manager.cc:1434] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:42.877215 27231 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.889166 27231 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.889832 27231 tablet_bootstrap.cc:492] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.890338 27231 ts_tablet_manager.cc:1403] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:20:42.892606 27231 raft_consensus.cc:359] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.893252 27231 raft_consensus.cc:740] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.893918 27231 consensus_queue.cc:260] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:42.895931 27231 ts_tablet_manager.cc:1434] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:20:42.896725 27231 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:42.908860 26438 consensus_peers.cc:597] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:42.922511 27231 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.923542 27231 tablet_bootstrap.cc:492] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.924136 27231 ts_tablet_manager.cc:1403] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.007s
I20260709 12:20:42.926127 27231 raft_consensus.cc:359] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.926690 27231 raft_consensus.cc:740] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.927592 27231 consensus_queue.cc:260] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.929597 27231 ts_tablet_manager.cc:1434] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:42.930485 27231 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.944494 27231 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.945248 27231 tablet_bootstrap.cc:492] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.945845 27231 ts_tablet_manager.cc:1403] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:20:42.947674 27231 raft_consensus.cc:359] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.948148 27231 raft_consensus.cc:740] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.948655 27231 consensus_queue.cc:260] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.950022 27231 ts_tablet_manager.cc:1434] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:42.950917 27231 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.964176 27231 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.965270 27231 tablet_bootstrap.cc:492] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.965950 27231 ts_tablet_manager.cc:1403] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:20:42.967971 27231 raft_consensus.cc:359] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.968472 27231 raft_consensus.cc:740] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.969075 27231 consensus_queue.cc:260] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:42.970950 27231 ts_tablet_manager.cc:1434] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:20:42.971930 27231 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:42.984328 27231 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:42.985087 27231 tablet_bootstrap.cc:492] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:42.985642 27231 ts_tablet_manager.cc:1403] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:20:42.987499 27231 raft_consensus.cc:359] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.988166 27231 raft_consensus.cc:740] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.988814 27231 consensus_queue.cc:260] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:42.990973 27231 ts_tablet_manager.cc:1434] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:42.991775 27231 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:42.997793 26438 consensus_peers.cc:597] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:43.006844 27231 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.007843 27231 tablet_bootstrap.cc:492] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.008517 27231 ts_tablet_manager.cc:1403] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.001s
I20260709 12:20:43.010967 27231 raft_consensus.cc:359] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.011600 27231 raft_consensus.cc:740] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.012218 27231 consensus_queue.cc:260] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.014449 27231 ts_tablet_manager.cc:1434] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:20:43.015333 27231 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.029654 27231 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.030435 27231 tablet_bootstrap.cc:492] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.031015 27231 ts_tablet_manager.cc:1403] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 12:20:43.033238 27231 raft_consensus.cc:359] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.033926 27231 raft_consensus.cc:740] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.034545 27231 consensus_queue.cc:260] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.036801 27231 ts_tablet_manager.cc:1434] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:43.037840 27231 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.051757 27231 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.052659 27231 tablet_bootstrap.cc:492] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.053328 27231 ts_tablet_manager.cc:1403] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.001s
I20260709 12:20:43.055615 27231 raft_consensus.cc:359] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.056301 27231 raft_consensus.cc:740] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.057071 27231 consensus_queue.cc:260] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.059573 27231 ts_tablet_manager.cc:1434] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:43.060379 27231 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.188781 27273 raft_consensus.cc:3060] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.178781 27271 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:43.260480 26438 consensus_peers.cc:597] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:43.314796 27231 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.316804 27269 raft_consensus.cc:3060] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.322516 27272 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:43.327303 26512 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 117407us
I20260709 12:20:43.360173 27274 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.381556 27231 tablet_bootstrap.cc:492] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.382788 27231 ts_tablet_manager.cc:1403] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.323s	user 0.048s	sys 0.045s
W20260709 12:20:43.455951 26512 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 77507.1us
I20260709 12:20:43.510061 27231 raft_consensus.cc:359] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.516219 27231 raft_consensus.cc:740] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.525213 27231 consensus_queue.cc:260] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.549031 27277 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.612695 27231 ts_tablet_manager.cc:1434] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.196s	user 0.005s	sys 0.007s
I20260709 12:20:43.614156 27231 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
W20260709 12:20:43.635542 27414 log.cc:927] Time spent T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.133s	user 0.004s	sys 0.000s
W20260709 12:20:43.673151 27423 log.cc:927] Time spent T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:20:43.692276 26438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 52592.2us
W20260709 12:20:43.702121 26441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:41389, user_credentials={real_user=slave}} blocked reactor thread for 54467.6us
W20260709 12:20:43.707584 26439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.68:44181, user_credentials={real_user=slave}} blocked reactor thread for 62881.4us
I20260709 12:20:43.716830 27276 raft_consensus.cc:3060] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.755802 27231 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.758077 27231 tablet_bootstrap.cc:492] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.758776 27231 ts_tablet_manager.cc:1403] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.145s	user 0.018s	sys 0.003s
I20260709 12:20:43.761521 27231 raft_consensus.cc:359] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:43.762751 27231 raft_consensus.cc:740] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.763445 27231 consensus_queue.cc:260] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:43.765344 27231 ts_tablet_manager.cc:1434] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 12:20:43.766144 27231 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.822044 27231 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.823042 27231 tablet_bootstrap.cc:492] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.823726 27231 ts_tablet_manager.cc:1403] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.005s
I20260709 12:20:43.826229 27231 raft_consensus.cc:359] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.826978 27231 raft_consensus.cc:740] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.827708 27231 consensus_queue.cc:260] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.831480 27231 ts_tablet_manager.cc:1434] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:20:43.832521 27231 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.860050 27231 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.861123 27231 tablet_bootstrap.cc:492] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.861831 27231 ts_tablet_manager.cc:1403] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.005s
I20260709 12:20:43.864329 27231 raft_consensus.cc:359] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.865003 27231 raft_consensus.cc:740] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.865868 27231 consensus_queue.cc:260] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.867991 27231 ts_tablet_manager.cc:1434] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:43.870445 27231 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.874055 27276 raft_consensus.cc:3060] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.892597 27231 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.893524 27231 tablet_bootstrap.cc:492] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.894107 27231 ts_tablet_manager.cc:1403] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.007s
I20260709 12:20:43.896349 27231 raft_consensus.cc:359] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.896935 27231 raft_consensus.cc:740] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.897549 27231 consensus_queue.cc:260] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.902034 27231 ts_tablet_manager.cc:1434] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:20:43.902868 27231 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.916669 27231 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.917557 27231 tablet_bootstrap.cc:492] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.918140 27231 ts_tablet_manager.cc:1403] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:20:43.920504 27231 raft_consensus.cc:359] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.921191 27231 raft_consensus.cc:740] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.921818 27231 consensus_queue.cc:260] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:43.923717 27231 ts_tablet_manager.cc:1434] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:20:43.924568 27231 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.942687 27231 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.943890 27231 tablet_bootstrap.cc:492] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.944340 27276 raft_consensus.cc:3060] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.944595 27231 ts_tablet_manager.cc:1403] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 12:20:43.947348 27231 raft_consensus.cc:359] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:43.948031 27231 raft_consensus.cc:740] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.955251 27231 consensus_queue.cc:260] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:43.957726 27231 ts_tablet_manager.cc:1434] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:20:43.958871 27231 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap starting.
I20260709 12:20:43.974321 27231 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:43.975371 27231 tablet_bootstrap.cc:492] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Bootstrap complete.
I20260709 12:20:43.976085 27231 ts_tablet_manager.cc:1403] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:20:43.978919 27231 raft_consensus.cc:359] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.979854 27231 raft_consensus.cc:740] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.980751 27231 consensus_queue.cc:260] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:43.983036 27231 ts_tablet_manager.cc:1434] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:44.198764 27277 raft_consensus.cc:3060] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:44.231631 27276 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:44.277577 27276 raft_consensus.cc:3060] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.312805 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.359405 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:51.360153 26614 raft_consensus.cc:606] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:51.362425 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:51.362910 26614 raft_consensus.cc:606] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:51.365501 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:51.365964 26614 raft_consensus.cc:606] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:51.368099 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "696254bba321492ab3ca55d7648e313b"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:51.368556 26614 raft_consensus.cc:606] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Received request to transfer leadership to TS 297900273c7740af9938749b3ce3a80d
I20260709 12:20:51.370776 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:51.371345 26614 raft_consensus.cc:606] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:51.381431 27448 raft_consensus.cc:993] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.381888 27448 raft_consensus.cc:1081] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.387420 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:51.388063 27277 raft_consensus.cc:493] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.388556 27277 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.394317 26474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:51.394853 26474 raft_consensus.cc:606] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:51.394959 27277 raft_consensus.cc:515] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.396061 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:20:51.397032 27277 leader_election.cc:290] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:51.397773 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:51.399045 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.414201 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:51.414969 26470 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.418148 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aa46be07aa5447cad5a356491417c4c" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:51.418771 26618 raft_consensus.cc:3055] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:51.419068 26618 raft_consensus.cc:740] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:51.419849 26618 consensus_queue.cc:260] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.421303 26618 raft_consensus.cc:3060] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.425082 26470 raft_consensus.cc:2468] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.428646 26618 raft_consensus.cc:2468] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.430982 27226 leader_election.cc:304] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:51.433408 27463 raft_consensus.cc:2804] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:51.434813 27463 raft_consensus.cc:697] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:51.436261 27463 consensus_queue.cc:237] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.448482 26294 catalog_manager.cc:5697] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.541589 27452 raft_consensus.cc:993] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.542070 27448 raft_consensus.cc:1081] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.543324 27274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:51.543781 27274 raft_consensus.cc:493] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.544153 27274 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.551365 27274 raft_consensus.cc:515] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.556667 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:51.557236 26614 raft_consensus.cc:3055] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:51.557510 26614 raft_consensus.cc:740] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:51.558336 27274 leader_election.cc:290] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:51.558212 26614 consensus_queue.cc:260] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.559417 26614 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.568360 27452 raft_consensus.cc:993] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.568826 27452 raft_consensus.cc:1081] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.569592 26614 raft_consensus.cc:2468] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.570446 27274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:51.570765 27226 leader_election.cc:304] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:51.571003 27274 raft_consensus.cc:493] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.571378 27274 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.571662 27463 raft_consensus.cc:2804] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:51.572203 27463 raft_consensus.cc:697] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:51.573169 27463 consensus_queue.cc:237] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.585477 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bbbf751e4544950af8ef067e8a1f70f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:51.586249 26545 raft_consensus.cc:3060] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.594172 26545 raft_consensus.cc:2468] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.597679 27274 raft_consensus.cc:515] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.599718 27274 leader_election.cc:290] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:51.599356 26294 catalog_manager.cc:5697] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:51.600723 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:51.601351 26543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8bad9126db041f7b83ae3e025d0c845" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:51.601984 26543 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.601358 26614 raft_consensus.cc:3055] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:51.607061 26543 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:51.607473 26614 raft_consensus.cc:740] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:51.608299 26614 consensus_queue.cc:260] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.609483 26614 raft_consensus.cc:3060] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.611564 27228 leader_election.cc:304] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:51.612382 27463 raft_consensus.cc:2804] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:51.612895 27463 raft_consensus.cc:697] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:51.613689 27463 consensus_queue.cc:237] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:51.615042 26614 raft_consensus.cc:2468] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:51.622056 26294 catalog_manager.cc:5697] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:51.722076 27417 raft_consensus.cc:993] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.722560 27417 raft_consensus.cc:1081] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.724174 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:51.724740 27277 raft_consensus.cc:493] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.725081 27277 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.728996 27448 raft_consensus.cc:993] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:51.729511 27471 raft_consensus.cc:1081] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Signalling peer 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:51.731070 26543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "696254bba321492ab3ca55d7648e313b"
dest_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:42982
I20260709 12:20:51.731559 26543 raft_consensus.cc:493] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.731349 27277 raft_consensus.cc:515] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:51.731930 26543 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.734472 27277 leader_election.cc:290] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:51.734812 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:51.735435 26470 raft_consensus.cc:3055] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:51.735738 26470 raft_consensus.cc:740] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:51.735697 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f76e69771f3c45fd842cbfca4e7b66d5" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:51.736476 26549 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.736743 26470 consensus_queue.cc:260] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:51.737718 26470 raft_consensus.cc:3060] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.739346 26543 raft_consensus.cc:515] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:51.742868 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:51.743432 26474 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.748602 26549 raft_consensus.cc:2468] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.749176 26470 raft_consensus.cc:2468] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:51.749837 26474 raft_consensus.cc:2468] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:51.750415 27228 leader_election.cc:304] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:51.751276 27463 raft_consensus.cc:2804] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:51.751457 26514 leader_election.cc:304] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:51.751806 27463 raft_consensus.cc:697] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:51.752403 27453 raft_consensus.cc:2804] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:51.752792 26543 leader_election.cc:290] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:51.752674 27463 consensus_queue.cc:237] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:51.753409 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "696254bba321492ab3ca55d7648e313b" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:51.753952 26614 raft_consensus.cc:3055] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:51.754249 26614 raft_consensus.cc:740] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:51.755002 26614 consensus_queue.cc:260] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:51.756184 26614 raft_consensus.cc:3060] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:51.759351 27453 raft_consensus.cc:697] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:51.760104 27453 consensus_queue.cc:237] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:51.766970 26614 raft_consensus.cc:2468] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:51.772687 26294 catalog_manager.cc:5697] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.772953 26298 catalog_manager.cc:5697] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.812685 27448 raft_consensus.cc:993] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.813184 27471 raft_consensus.cc:1081] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:51.814546 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:51.815039 27277 raft_consensus.cc:493] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:51.815663 27277 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.821480 27277 raft_consensus.cc:515] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.823385 27277 leader_election.cc:290] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:51.827136 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:51.827692 26474 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.827883 26618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "113f3dbb8bd5466abbfb043ef9cc4d87" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:51.828444 26618 raft_consensus.cc:3055] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:51.828738 26618 raft_consensus.cc:740] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:51.829510 26618 consensus_queue.cc:260] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.830732 26618 raft_consensus.cc:3060] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:51.834085 26474 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:51.835657 27227 leader_election.cc:304] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:51.839443 26618 raft_consensus.cc:2468] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:51.840211 27463 raft_consensus.cc:2804] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:51.840755 27463 raft_consensus.cc:697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:51.841503 27463 consensus_queue.cc:237] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:51.851711 26298 catalog_manager.cc:5697] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.859844 26474 raft_consensus.cc:1275] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:51.861295 27463 consensus_queue.cc:1048] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:51.874179 26618 raft_consensus.cc:1275] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:51.875677 27463 consensus_queue.cc:1048] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:52.049301 26543 raft_consensus.cc:1275] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.050534 27463 consensus_queue.cc:1048] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:52.061328 26614 raft_consensus.cc:1275] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.065282 26545 raft_consensus.cc:1275] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.066793 27463 consensus_queue.cc:1048] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.092896 26614 raft_consensus.cc:1275] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.088819 27463 consensus_queue.cc:1048] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:52.094681 27484 consensus_queue.cc:1048] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.155308 26545 raft_consensus.cc:1275] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.157058 27484 consensus_queue.cc:1048] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.182078 26470 raft_consensus.cc:1275] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.183351 27463 consensus_queue.cc:1048] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:52.211835 26618 raft_consensus.cc:1275] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.213316 27449 consensus_queue.cc:1048] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:52.257155 26470 raft_consensus.cc:1275] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:52.259517 27490 consensus_queue.cc:1048] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.324712 26614 raft_consensus.cc:1275] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.326011 27484 consensus_queue.cc:1048] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:52.335922 26470 raft_consensus.cc:1275] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.337196 27484 consensus_queue.cc:1048] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:52.455288 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.498805 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:52.499468 26614 raft_consensus.cc:606] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:52.505563 26470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:52.506141 26470 raft_consensus.cc:606] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS 297900273c7740af9938749b3ce3a80d
I20260709 12:20:52.508668 26470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:52.509191 26470 raft_consensus.cc:606] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:52.515086 26470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb49253f88584c74979fb16378c007d1"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:52.515667 26470 raft_consensus.cc:606] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS 297900273c7740af9938749b3ce3a80d
I20260709 12:20:52.517781 26470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:52.518196 26470 raft_consensus.cc:606] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:52.519129 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:52.520316 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:52.520889 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:52.572412 27417 raft_consensus.cc:993] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.573035 27454 raft_consensus.cc:1081] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.574572 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:52.575191 27277 raft_consensus.cc:493] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:52.575611 27277 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.580618 27277 raft_consensus.cc:515] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.582154 27277 leader_election.cc:290] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:52.583106 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:52.583194 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d729d1bc559949d28af8e08f5a719332" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:52.583834 26549 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.583856 26470 raft_consensus.cc:3055] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:52.584455 26470 raft_consensus.cc:740] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:52.585182 26470 consensus_queue.cc:260] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.586107 26470 raft_consensus.cc:3060] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.589035 26549 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:52.589994 27228 leader_election.cc:304] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:52.590644 27484 raft_consensus.cc:2804] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:52.590943 26470 raft_consensus.cc:2468] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:52.591169 27484 raft_consensus.cc:697] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:52.592106 27484 consensus_queue.cc:237] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.602891 26294 catalog_manager.cc:5697] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.730803 27448 raft_consensus.cc:993] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.731370 27488 raft_consensus.cc:1081] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.732780 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:52.733280 27277 raft_consensus.cc:493] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:52.733562 27277 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.738073 27277 raft_consensus.cc:515] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:52.739851 27277 leader_election.cc:290] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:52.740540 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:52.740769 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56bca68ee09b4f00a06f68b9fc9ca949" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:52.741676 26549 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.741755 26614 raft_consensus.cc:3055] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:52.742187 26614 raft_consensus.cc:740] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:52.742848 26614 consensus_queue.cc:260] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:52.743932 26614 raft_consensus.cc:3060] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.747879 26549 raft_consensus.cc:2468] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:52.749722 26614 raft_consensus.cc:2468] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:52.749872 27228 leader_election.cc:304] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:52.750661 27463 raft_consensus.cc:2804] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:52.751175 27463 raft_consensus.cc:697] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:52.752043 27463 consensus_queue.cc:237] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:52.765455 26294 catalog_manager.cc:5697] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:52.897083 27454 raft_consensus.cc:993] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: : Instructing follower 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:52.897552 27460 raft_consensus.cc:1081] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:52.898795 26549 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca"
dest_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:42968
I20260709 12:20:52.899271 26549 raft_consensus.cc:493] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:52.899575 26549 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.903878 26549 raft_consensus.cc:515] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:52.906172 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:52.906735 26549 leader_election.cc:290] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:52.907174 27454 raft_consensus.cc:993] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.907338 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cc5473d41af4ddc9dd47559842556ca" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:52.907682 27454 raft_consensus.cc:1081] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:52.907887 26614 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.906700 26470 raft_consensus.cc:3055] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:52.908321 26470 raft_consensus.cc:740] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:52.909219 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:52.909082 26470 consensus_queue.cc:260] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:52.909894 27277 raft_consensus.cc:493] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:52.910356 26470 raft_consensus.cc:3060] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:52.910429 27277 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.913133 26614 raft_consensus.cc:2468] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:52.914033 26513 leader_election.cc:304] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:52.914680 27494 raft_consensus.cc:2804] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:52.915105 27494 raft_consensus.cc:697] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:52.915952 27494 consensus_queue.cc:237] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:52.916522 26470 raft_consensus.cc:2468] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:52.917637 27277 raft_consensus.cc:515] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.919827 27277 leader_election.cc:290] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:52.921120 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:52.921721 26549 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.924957 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad43a300a529407993bd1819f2e09aaa" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:52.925539 26470 raft_consensus.cc:3055] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:52.925828 26470 raft_consensus.cc:740] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:52.926533 26470 consensus_queue.cc:260] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.927659 26470 raft_consensus.cc:3060] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:52.927986 26549 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:52.929013 27228 leader_election.cc:304] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:52.928948 26294 catalog_manager.cc:5697] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.929858 27463 raft_consensus.cc:2804] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:52.930428 27463 raft_consensus.cc:697] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:52.931257 27463 consensus_queue.cc:237] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:52.932986 26470 raft_consensus.cc:2468] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:52.942860 26294 catalog_manager.cc:5697] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:53.034247 27454 raft_consensus.cc:993] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: : Instructing follower 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:53.034732 27460 raft_consensus.cc:1081] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:53.036331 26549 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb49253f88584c74979fb16378c007d1"
dest_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:42968
I20260709 12:20:53.036859 26549 raft_consensus.cc:493] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:53.037158 26549 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.043164 26549 raft_consensus.cc:515] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.045096 26549 leader_election.cc:290] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:53.045361 26470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:53.045944 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb49253f88584c74979fb16378c007d1" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:53.046006 26470 raft_consensus.cc:3055] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:53.046509 26614 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.046692 26470 raft_consensus.cc:740] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:53.047473 26470 consensus_queue.cc:260] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.048511 26470 raft_consensus.cc:3060] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.052889 26614 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:53.054147 26513 leader_election.cc:304] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:53.055097 27503 raft_consensus.cc:2804] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:53.055379 26470 raft_consensus.cc:2468] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:53.055622 27503 raft_consensus.cc:697] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:53.057096 27503 consensus_queue.cc:237] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.078410 26294 catalog_manager.cc:5697] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:53.195670 26549 raft_consensus.cc:1275] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.201421 27483 consensus_queue.cc:1048] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:20:53.221693 26543 raft_consensus.cc:1275] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:53.228725 27484 consensus_queue.cc:1048] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:20:53.254458 26586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:41581, user_credentials={real_user=slave}} blocked reactor thread for 47082us
I20260709 12:20:53.284477 26470 raft_consensus.cc:1275] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:53.306524 27512 consensus_queue.cc:1048] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:53.371502 26614 raft_consensus.cc:1275] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.378975 27503 consensus_queue.cc:1048] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:53.382687 26618 raft_consensus.cc:1275] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.385522 26470 raft_consensus.cc:1275] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.386524 27512 consensus_queue.cc:1048] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:53.395736 27490 consensus_queue.cc:1048] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:53.406332 26470 raft_consensus.cc:1275] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:53.425580 27512 consensus_queue.cc:1048] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:20:53.477201 26545 raft_consensus.cc:1275] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:53.480960 26474 raft_consensus.cc:1275] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.491837 27484 consensus_queue.cc:1048] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:53.498054 27494 consensus_queue.cc:1048] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:53.562546 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.685196 26614 raft_consensus.cc:1275] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:53.696681 27503 consensus_queue.cc:1048] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:53.746548 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:53.747381 26471 raft_consensus.cc:606] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS 297900273c7740af9938749b3ce3a80d
I20260709 12:20:53.750648 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:53.751323 26471 raft_consensus.cc:606] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:53.753857 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:53.754347 26471 raft_consensus.cc:606] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:53.756592 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "39930cd9790d4943bdec6f9df4ce2641"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:53.757074 26471 raft_consensus.cc:606] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:53.759344 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce90f860c124766af2e9d46c5864016"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:53.759810 26471 raft_consensus.cc:606] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:53.761610 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:53.762802 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:53.763561 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.913945 27510 raft_consensus.cc:993] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: : Instructing follower 39930cd9790d4943bdec6f9df4ce2641 to start an election
I20260709 12:20:53.914692 27454 raft_consensus.cc:1081] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer 39930cd9790d4943bdec6f9df4ce2641 to start an election
I20260709 12:20:53.916354 26614 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
 from {username='slave'} at 127.0.0.1:45454
I20260709 12:20:53.917013 26614 raft_consensus.cc:493] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:53.917366 26614 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.926257 26614 raft_consensus.cc:515] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.928254 26614 leader_election.cc:290] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:53.929275 26545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:53.929148 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f838e26a3f48ebb76d7b98527fe2d0" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:53.929960 26545 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.930066 26471 raft_consensus.cc:3055] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:53.930557 26471 raft_consensus.cc:740] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:53.931442 26471 consensus_queue.cc:260] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.932780 26471 raft_consensus.cc:3060] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:53.938673 26545 raft_consensus.cc:2468] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:53.940120 26589 leader_election.cc:304] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, 39930cd9790d4943bdec6f9df4ce2641; no voters: 
I20260709 12:20:53.941193 27511 raft_consensus.cc:2804] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:53.941627 27511 raft_consensus.cc:697] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:53.942685 27511 consensus_queue.cc:237] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:53.951785 26471 raft_consensus.cc:2468] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:53.958658 26297 catalog_manager.cc:5697] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.061920 27454 raft_consensus.cc:993] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.062490 27454 raft_consensus.cc:1081] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.065378 27276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bce90f860c124766af2e9d46c5864016"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:54.065929 27276 raft_consensus.cc:493] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.066295 27276 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.075174 27276 raft_consensus.cc:515] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:54.077467 27276 leader_election.cc:290] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:54.079542 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:54.080180 26549 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.081245 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce90f860c124766af2e9d46c5864016" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:54.081787 26474 raft_consensus.cc:3055] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:54.081969 26474 raft_consensus.cc:740] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:54.082530 26474 consensus_queue.cc:260] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:54.083817 26474 raft_consensus.cc:3060] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.090932 26549 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:54.092293 27228 leader_election.cc:304] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:54.092300 26474 raft_consensus.cc:2468] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:54.093051 27484 raft_consensus.cc:2804] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.093540 27484 raft_consensus.cc:697] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:54.094440 27484 consensus_queue.cc:237] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:54.105743 26297 catalog_manager.cc:5697] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.114475 27514 raft_consensus.cc:993] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: : Instructing follower 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:54.114941 27454 raft_consensus.cc:1081] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer 297900273c7740af9938749b3ce3a80d to start an election
I20260709 12:20:54.116554 26546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6"
dest_uuid: "297900273c7740af9938749b3ce3a80d"
 from {username='slave'} at 127.0.0.1:42968
I20260709 12:20:54.117110 26546 raft_consensus.cc:493] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.117494 26546 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.124104 26546 raft_consensus.cc:515] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:54.125944 26546 leader_election.cc:290] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:54.126595 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:54.127316 26474 raft_consensus.cc:3055] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:54.127481 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0822222bd071434cb1c7ec59559e02a6" candidate_uuid: "297900273c7740af9938749b3ce3a80d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:54.127712 26474 raft_consensus.cc:740] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:54.128284 26617 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.128885 26474 consensus_queue.cc:260] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:54.130326 26474 raft_consensus.cc:3060] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.132373 27514 raft_consensus.cc:993] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.132936 27454 raft_consensus.cc:1081] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.135095 27276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:54.135686 27276 raft_consensus.cc:493] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.136065 27276 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.136148 26474 raft_consensus.cc:2468] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:54.136791 26617 raft_consensus.cc:2468] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 297900273c7740af9938749b3ce3a80d in term 2.
I20260709 12:20:54.137368 26514 leader_election.cc:304] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:54.138314 27530 raft_consensus.cc:2804] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.138792 27530 raft_consensus.cc:697] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Becoming Leader. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:54.139724 27530 consensus_queue.cc:237] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:54.149435 27276 raft_consensus.cc:515] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.154490 27514 raft_consensus.cc:993] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.155050 27510 raft_consensus.cc:1081] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.155256 27276 leader_election.cc:290] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:54.155133 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:54.155979 26617 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.156387 26474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4358c31532834ddd8a1e690e2accadfb" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:54.156870 27276 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:54.157058 26474 raft_consensus.cc:3055] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:54.157493 27276 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.158056 27276 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:54.157521 26474 raft_consensus.cc:740] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:54.163816 26617 raft_consensus.cc:2468] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:54.164588 26297 catalog_manager.cc:5697] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to 297900273c7740af9938749b3ce3a80d (127.16.193.67). New cstate: current_term: 2 leader_uuid: "297900273c7740af9938749b3ce3a80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.165839 27226 leader_election.cc:304] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:54.166680 27484 raft_consensus.cc:2804] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.167163 27484 raft_consensus.cc:697] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:54.167460 27276 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.161743 26474 consensus_queue.cc:260] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.168267 27484 consensus_queue.cc:237] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.171149 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:54.171768 26471 raft_consensus.cc:3055] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:54.172050 26471 raft_consensus.cc:740] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:54.172976 26471 consensus_queue.cc:260] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.174219 26471 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:54.177392 26474 raft_consensus.cc:3060] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.170624 27276 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 4 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:54.184064 26471 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 4.
I20260709 12:20:54.186163 27227 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:54.187099 27523 raft_consensus.cc:2804] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:54.187664 27523 raft_consensus.cc:697] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 4 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:54.188560 27523 consensus_queue.cc:237] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:54.173267 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "948eb78b01d544e6a0e64f60b261edea" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:54.191543 26617 raft_consensus.cc:3060] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:54.196385 26474 raft_consensus.cc:2468] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:54.218856 26297 catalog_manager.cc:5697] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.225953 26617 raft_consensus.cc:2468] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 4.
I20260709 12:20:54.238024 26297 catalog_manager.cc:5697] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 3 to 4, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 4 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.375244 26474 raft_consensus.cc:1275] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.376374 27511 consensus_queue.cc:1048] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.391142 26546 raft_consensus.cc:1275] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.392620 27511 consensus_queue.cc:1048] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.569990 26546 raft_consensus.cc:1275] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.571424 27484 consensus_queue.cc:1048] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.583897 26474 raft_consensus.cc:1275] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.585295 27523 consensus_queue.cc:1048] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.634614 26617 raft_consensus.cc:1275] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:54.635908 27523 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.646773 26471 raft_consensus.cc:1275] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 4 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:54.648202 27463 consensus_queue.cc:1048] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.650036 26474 raft_consensus.cc:1275] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.652367 27490 consensus_queue.cc:1048] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.669667 26614 raft_consensus.cc:1275] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer 297900273c7740af9938749b3ce3a80d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.671262 27503 consensus_queue.cc:1048] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.722559 26614 raft_consensus.cc:1275] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.723912 27463 consensus_queue.cc:1048] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.734086 26474 raft_consensus.cc:1275] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:54.735364 27523 consensus_queue.cc:1048] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:54.799070 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.861326 26549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b"
dest_uuid: "297900273c7740af9938749b3ce3a80d"
mode: GRACEFUL
new_leader_uuid: "39930cd9790d4943bdec6f9df4ce2641"
 from {username='slave'} at 127.0.0.1:34928
I20260709 12:20:54.861797 26549 raft_consensus.cc:606] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Received request to transfer leadership to TS 39930cd9790d4943bdec6f9df4ce2641
I20260709 12:20:54.864426 26471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f"
dest_uuid: "bb83296e641848deaddf424cc59fda0e"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:46702
I20260709 12:20:54.864871 26471 raft_consensus.cc:606] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:54.867280 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf0a016521b542548a73d6110ed04210"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:54.867724 26614 raft_consensus.cc:606] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:54.868865 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:54.870023 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:54.870610 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:54.902513 27448 raft_consensus.cc:993] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.903061 27448 raft_consensus.cc:1081] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:54.904771 27274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf0a016521b542548a73d6110ed04210"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:54.905351 27274 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:54.905650 27274 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.911562 27274 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:54.913038 27274 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181)
I20260709 12:20:54.913774 26549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:54.913959 26614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0a016521b542548a73d6110ed04210" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:54.914465 26549 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.914479 26614 raft_consensus.cc:3055] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:54.915019 26614 raft_consensus.cc:740] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:54.915896 26614 consensus_queue.cc:260] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:54.917088 26614 raft_consensus.cc:3060] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.921293 26549 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:54.922632 27228 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:54.923426 27523 raft_consensus.cc:2804] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:54.923871 27523 raft_consensus.cc:697] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:54.923933 26614 raft_consensus.cc:2468] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:54.924850 27523 consensus_queue.cc:237] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:54.936077 26297 catalog_manager.cc:5697] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:55.213575 27530 raft_consensus.cc:993] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: : Instructing follower 39930cd9790d4943bdec6f9df4ce2641 to start an election
I20260709 12:20:55.214138 27490 raft_consensus.cc:1081] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Signalling peer 39930cd9790d4943bdec6f9df4ce2641 to start an election
I20260709 12:20:55.215530 26614 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
 from {username='slave'} at 127.0.0.1:45450
I20260709 12:20:55.215993 26614 raft_consensus.cc:493] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:55.216287 26614 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.220932 26614 raft_consensus.cc:515] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:55.222321 26614 leader_election.cc:290] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:55.223026 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:55.223189 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157df6bf4a63423eb6f1c4f7de58fb9b" candidate_uuid: "39930cd9790d4943bdec6f9df4ce2641" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:55.223752 26471 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.223825 26546 raft_consensus.cc:3055] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:55.224231 26546 raft_consensus.cc:740] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 297900273c7740af9938749b3ce3a80d, State: Running, Role: LEADER
I20260709 12:20:55.224926 26546 consensus_queue.cc:260] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:55.225953 26546 raft_consensus.cc:3060] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.229724 26471 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:55.230963 26546 raft_consensus.cc:2468] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39930cd9790d4943bdec6f9df4ce2641 in term 2.
I20260709 12:20:55.231025 26588 leader_election.cc:304] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641, bb83296e641848deaddf424cc59fda0e; no voters: 
I20260709 12:20:55.232136 27511 raft_consensus.cc:2804] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.232836 27511 raft_consensus.cc:697] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Leader. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:55.234333 27511 consensus_queue.cc:237] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } }
I20260709 12:20:55.245012 26297 catalog_manager.cc:5697] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 reported cstate change: term changed from 1 to 2, leader changed from 297900273c7740af9938749b3ce3a80d (127.16.193.67) to 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68). New cstate: current_term: 2 leader_uuid: "39930cd9790d4943bdec6f9df4ce2641" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:55.316933 26546 raft_consensus.cc:1275] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:55.318094 27523 consensus_queue.cc:1048] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:55.331055 26614 raft_consensus.cc:1275] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:55.332376 27523 consensus_queue.cc:1048] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:55.448088 27514 raft_consensus.cc:993] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:55.448688 27454 raft_consensus.cc:1081] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:55.450193 27277 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54930
I20260709 12:20:55.450611 27277 raft_consensus.cc:493] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:55.450851 27277 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.455968 27277 raft_consensus.cc:515] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:55.457257 27277 leader_election.cc:290] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Requested vote from peers bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205), 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389)
I20260709 12:20:55.458230 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:55.458297 26546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b91ba46bcd3e4802a197552bcbda449f" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "297900273c7740af9938749b3ce3a80d"
I20260709 12:20:55.458909 26471 raft_consensus.cc:3055] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:55.458987 26546 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.459277 26471 raft_consensus.cc:740] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb83296e641848deaddf424cc59fda0e, State: Running, Role: LEADER
I20260709 12:20:55.460387 26471 consensus_queue.cc:260] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:55.461762 26471 raft_consensus.cc:3060] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.465483 26546 raft_consensus.cc:2468] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:55.466482 27228 leader_election.cc:304] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:55.467185 27523 raft_consensus.cc:2804] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.467718 27523 raft_consensus.cc:697] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:55.468747 27523 consensus_queue.cc:237] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } }
I20260709 12:20:55.469182 26471 raft_consensus.cc:2468] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 2.
I20260709 12:20:55.479177 26297 catalog_manager.cc:5697] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 1 to 2, leader changed from bb83296e641848deaddf424cc59fda0e (127.16.193.66) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.697410 26471 raft_consensus.cc:1275] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.698698 27511 consensus_queue.cc:1048] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.710968 26546 raft_consensus.cc:1275] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer 39930cd9790d4943bdec6f9df4ce2641: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.712704 27569 consensus_queue.cc:1048] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.906962 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:55.947162 26614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291"
dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
mode: GRACEFUL
new_leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:48420
I20260709 12:20:55.947829 26614 raft_consensus.cc:606] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Received request to transfer leadership to TS fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:55.949016 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:20:55.950357 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:55.950919 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:56.047802 26549 raft_consensus.cc:1275] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:56.049280 27523 consensus_queue.cc:1048] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:56.063337 26471 raft_consensus.cc:1275] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:56.064914 27463 consensus_queue.cc:1048] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:56.410928 27576 raft_consensus.cc:993] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: : Instructing follower fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:56.411442 27576 raft_consensus.cc:1081] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Signalling peer fc524c91d3c249c0aeb08dae83b98e41 to start an election
I20260709 12:20:56.412755 27274 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291"
dest_uuid: "fc524c91d3c249c0aeb08dae83b98e41"
 from {username='slave'} at 127.0.0.1:54948
I20260709 12:20:56.413200 27274 raft_consensus.cc:493] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.413491 27274 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:56.419929 27274 raft_consensus.cc:515] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:56.422133 27274 leader_election.cc:290] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Requested vote from peers 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68:44181), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:56.423760 26617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39930cd9790d4943bdec6f9df4ce2641"
I20260709 12:20:56.424420 26471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4125b700ff114ebeb4ff2849edf5e291" candidate_uuid: "fc524c91d3c249c0aeb08dae83b98e41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb83296e641848deaddf424cc59fda0e"
I20260709 12:20:56.424505 26617 raft_consensus.cc:3055] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:56.425297 26471 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:56.425482 26617 raft_consensus.cc:740] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39930cd9790d4943bdec6f9df4ce2641, State: Running, Role: LEADER
I20260709 12:20:56.426448 26617 consensus_queue.cc:260] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:56.427848 26617 raft_consensus.cc:3060] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:56.435600 26471 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:56.436766 27227 leader_election.cc:304] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41; no voters: 
I20260709 12:20:56.437608 27523 raft_consensus.cc:2804] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:56.438050 27523 raft_consensus.cc:697] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Becoming Leader. State: Replica: fc524c91d3c249c0aeb08dae83b98e41, State: Running, Role: LEADER
I20260709 12:20:56.438156 26617 raft_consensus.cc:2468] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc524c91d3c249c0aeb08dae83b98e41 in term 3.
I20260709 12:20:56.439018 27523 consensus_queue.cc:237] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:56.461467 26297 catalog_manager.cc:5697] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 reported cstate change: term changed from 2 to 3, leader changed from 39930cd9790d4943bdec6f9df4ce2641 (127.16.193.68) to fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65). New cstate: current_term: 3 leader_uuid: "fc524c91d3c249c0aeb08dae83b98e41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:56.893996 26614 raft_consensus.cc:1275] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:56.895264 27463 consensus_queue.cc:1048] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:56.911285 26471 raft_consensus.cc:1275] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Refusing update from remote peer fc524c91d3c249c0aeb08dae83b98e41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:56.913199 27581 consensus_queue.cc:1048] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:56.984452 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:57.020390 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:57.021353 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:57.021965 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:57.056838 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:57.094666 17157 tablet_server.cc:179] TabletServer@127.16.193.65:41581 shutting down...
W20260709 12:20:57.118698 26438 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 62 similar messages]
W20260709 12:20:57.124670 26438 consensus_peers.cc:597] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.129393 26438 consensus_peers.cc:597] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.129408 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:57.130934 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.132298 17157 raft_consensus.cc:2243] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.132980 17157 raft_consensus.cc:2272] T a97fd66be7ff4a588ce7be8aa1a5b5be P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.136612 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.138628 17157 raft_consensus.cc:2243] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.139349 17157 raft_consensus.cc:2272] T 237bd2b722334bda9aeb29035546ca4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.142527 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.143561 17157 raft_consensus.cc:2243] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.144711 17157 raft_consensus.cc:2272] T 113f3dbb8bd5466abbfb043ef9cc4d87 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.154593 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.155488 17157 raft_consensus.cc:2243] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.156563 17157 raft_consensus.cc:2272] T 4125b700ff114ebeb4ff2849edf5e291 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.157207 26586 consensus_peers.cc:597] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.159199 26438 consensus_peers.cc:597] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.162251 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.163218 17157 raft_consensus.cc:2243] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.163823 17157 raft_consensus.cc:2272] T 52d9fdb0a26d4b43a1940f3ca15c530f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.166458 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.167325 17157 raft_consensus.cc:2243] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.168378 17157 raft_consensus.cc:2272] T ad43a300a529407993bd1819f2e09aaa P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.171044 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.171928 17157 raft_consensus.cc:2243] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.172369 17157 raft_consensus.cc:2272] T 759e0f60b4604c2e9abda3833a0e9e6f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.175352 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.176051 17157 raft_consensus.cc:2243] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.176883 17157 raft_consensus.cc:2272] T d729d1bc559949d28af8e08f5a719332 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.178995 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.179600 17157 raft_consensus.cc:2243] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.179996 17157 raft_consensus.cc:2272] T f94d403c93084635b0e4c70ec3829e1c P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.182698 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.183308 17157 raft_consensus.cc:2243] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.183765 17157 raft_consensus.cc:2272] T b00aa72f3c874a77b2d226ecbebbe834 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.185740 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.186265 17157 raft_consensus.cc:2243] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.187011 17157 raft_consensus.cc:2272] T bce90f860c124766af2e9d46c5864016 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.189433 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.190248 17157 raft_consensus.cc:2243] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.191058 17157 raft_consensus.cc:2272] T 56bca68ee09b4f00a06f68b9fc9ca949 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.193740 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.194293 17157 raft_consensus.cc:2243] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.194803 17157 raft_consensus.cc:2272] T d7cc64cd1a8348f68f5b9115f9205e40 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.197129 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.197841 17157 raft_consensus.cc:2243] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.198458 17157 raft_consensus.cc:2272] T d6eeaddd79764e06b3ac77e3c20ffd49 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.199503 26512 consensus_peers.cc:597] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.201258 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.201893 17157 raft_consensus.cc:2243] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.202306 17157 raft_consensus.cc:2272] T 429d15dc784e4494ad111c92f4812cc6 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.204540 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.205262 17157 raft_consensus.cc:2243] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.205747 17157 raft_consensus.cc:2272] T 81dffbd4eb274c0595db3619e8eb2058 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.208382 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.208937 17157 raft_consensus.cc:2243] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.209415 17157 raft_consensus.cc:2272] T c141b67bfa314acca1c0c3c29d3eaac8 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.212064 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.212844 17157 raft_consensus.cc:2243] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.213188 17157 raft_consensus.cc:2272] T 8f22e34b989c46e1867151870954a0e5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.215021 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.215726 17157 raft_consensus.cc:2243] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.216208 17157 raft_consensus.cc:2272] T 899048d400bc4540a891c445bcd9bd25 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.218631 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.219194 17157 raft_consensus.cc:2243] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.219745 17157 raft_consensus.cc:2272] T f97b53680d1349fa8b2bb3c5e067602d P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.222683 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.223608 17157 raft_consensus.cc:2243] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.224095 17157 raft_consensus.cc:2272] T eff188538952450b970105de07be9a85 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.226548 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.227094 17157 raft_consensus.cc:2243] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.227982 17157 raft_consensus.cc:2272] T d8bad9126db041f7b83ae3e025d0c845 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.229892 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.230481 17157 raft_consensus.cc:2243] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.230965 17157 raft_consensus.cc:2272] T 235d0175b299499399baaa177b25de35 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.233127 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.233668 17157 raft_consensus.cc:2243] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.234105 17157 raft_consensus.cc:2272] T 84283d0aa91944d482d29b9d70c80299 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.236377 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.237017 17157 raft_consensus.cc:2243] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.237391 17157 raft_consensus.cc:2272] T 9f6c90924ba74445ae2edc110c400e78 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.239475 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.240103 17157 raft_consensus.cc:2243] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.240447 17157 raft_consensus.cc:2272] T 966ed0b8df3447849f8bccb9b1af5b33 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.242408 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.242906 17157 raft_consensus.cc:2243] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.243481 17157 raft_consensus.cc:2272] T df382f80de9947869b3760fc62e30a77 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.245824 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.246376 17157 raft_consensus.cc:2243] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.247188 17157 raft_consensus.cc:2272] T f76e69771f3c45fd842cbfca4e7b66d5 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.249130 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.249621 17157 raft_consensus.cc:2243] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.249964 17157 raft_consensus.cc:2272] T 8fc216f2a251463a9c406ab418b62360 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.252012 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.252434 17157 raft_consensus.cc:2243] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.252797 17157 raft_consensus.cc:2272] T 97abcb3c90494ee2b8ad8f45c34923f5 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.254719 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.255204 17157 raft_consensus.cc:2243] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.256047 17157 raft_consensus.cc:2272] T bf0a016521b542548a73d6110ed04210 P fc524c91d3c249c0aeb08dae83b98e41 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.258001 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.258477 17157 raft_consensus.cc:2243] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.259212 17157 raft_consensus.cc:2272] T 4aa46be07aa5447cad5a356491417c4c P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.261097 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.261536 17157 raft_consensus.cc:2243] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.262183 17157 raft_consensus.cc:2272] T 948eb78b01d544e6a0e64f60b261edea P fc524c91d3c249c0aeb08dae83b98e41 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.264912 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.265688 17157 raft_consensus.cc:2243] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.266556 17157 raft_consensus.cc:2272] T 4358c31532834ddd8a1e690e2accadfb P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.269304 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.270084 17157 raft_consensus.cc:2243] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:57.270085 26438 consensus_peers.cc:597] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.270772 17157 raft_consensus.cc:2272] T ba40df2a50424f46b52f9c4cf6894664 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.273902 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.274623 17157 raft_consensus.cc:2243] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.275055 17157 raft_consensus.cc:2272] T 095ead81e4364b36a759035a2def0683 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.277329 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.277864 17157 raft_consensus.cc:2243] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.278329 17157 raft_consensus.cc:2272] T c6ee0ada749043888b2a134a520a25e1 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.280467 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.280938 17157 raft_consensus.cc:2243] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.281399 17157 raft_consensus.cc:2272] T 86ba667160b44ddf81a0d48eedb7bdae P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.283736 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.284428 17157 raft_consensus.cc:2243] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.285305 17157 raft_consensus.cc:2272] T b91ba46bcd3e4802a197552bcbda449f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.287824 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.288446 17157 raft_consensus.cc:2243] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.288890 17157 raft_consensus.cc:2272] T fe3aff16fd4242f5a9ef1a3038bbe22f P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.290320 26586 consensus_peers.cc:597] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.291419 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.291997 17157 raft_consensus.cc:2243] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.292389 17157 raft_consensus.cc:2272] T 32add6893dda4c028a237f05c1f1ec81 P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.294519 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.295019 17157 raft_consensus.cc:2243] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.295460 17157 raft_consensus.cc:2272] T a289521cf13242cf80f6cf90c53b8ab4 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.297746 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.298269 17157 raft_consensus.cc:2243] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.299260 17157 raft_consensus.cc:2272] T 3bbbf751e4544950af8ef067e8a1f70f P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.301882 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.302552 17157 raft_consensus.cc:2243] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.303082 17157 raft_consensus.cc:2272] T 063cdd050c3d40fdab471dec902e20e7 P fc524c91d3c249c0aeb08dae83b98e41 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.305153 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:20:57.305555 17157 raft_consensus.cc:2243] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.305943 17157 raft_consensus.cc:2272] T 85165efa340d47638ef75f49dda75e8a P fc524c91d3c249c0aeb08dae83b98e41 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.342090 26438 consensus_peers.cc:597] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.346500 26438 consensus_peers.cc:597] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.353705 26586 consensus_peers.cc:597] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.354851 26512 consensus_peers.cc:597] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.361042 26438 consensus_peers.cc:597] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.377319 26586 consensus_peers.cc:597] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.386293 17157 tablet_server.cc:196] TabletServer@127.16.193.65:41581 shutdown complete.
W20260709 12:20:57.416715 26512 consensus_peers.cc:597] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.434685 26512 consensus_peers.cc:597] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.435173 26512 consensus_peers.cc:597] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.445406 26438 consensus_peers.cc:597] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.458778 26586 consensus_peers.cc:597] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.464035 26438 consensus_peers.cc:597] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.484387 26438 consensus_peers.cc:597] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.492852 26586 consensus_peers.cc:597] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.494555 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
W20260709 12:20:57.508734 26586 consensus_peers.cc:597] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.512727 26438 consensus_peers.cc:597] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.519848 26588 consensus_peers.cc:597] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.529074 26512 consensus_peers.cc:597] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.531524 26514 consensus_peers.cc:597] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.544842 26588 consensus_peers.cc:597] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.546129 26512 consensus_peers.cc:597] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.548822 26514 consensus_peers.cc:597] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.553659 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:57.554777 17157 tablet_replica.cc:333] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.555665 17157 raft_consensus.cc:2243] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.556722 17157 raft_consensus.cc:2272] T 063cdd050c3d40fdab471dec902e20e7 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.559056 17157 tablet_replica.cc:333] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.559913 17157 raft_consensus.cc:2243] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.560994 17157 raft_consensus.cc:2272] T 2dcf2bebeaf8451e890bcbe46a6ae967 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.564447 17157 tablet_replica.cc:333] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.565320 17157 raft_consensus.cc:2243] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:57.565452 26586 consensus_peers.cc:597] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.566118 17157 raft_consensus.cc:2272] T 235d0175b299499399baaa177b25de35 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.568593 17157 tablet_replica.cc:333] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.569326 17157 raft_consensus.cc:2243] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.569772 17157 raft_consensus.cc:2272] T 696254bba321492ab3ca55d7648e313b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.571838 17157 tablet_replica.cc:333] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.572479 17157 raft_consensus.cc:2243] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.573535 17157 raft_consensus.cc:2272] T d7cc64cd1a8348f68f5b9115f9205e40 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.576067 17157 tablet_replica.cc:333] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.577005 17157 raft_consensus.cc:2243] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.577613 17157 raft_consensus.cc:2272] T a97fd66be7ff4a588ce7be8aa1a5b5be P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.580672 17157 tablet_replica.cc:333] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.581503 17157 raft_consensus.cc:2243] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.582046 17157 raft_consensus.cc:2272] T a9f838e26a3f48ebb76d7b98527fe2d0 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.585453 17157 tablet_replica.cc:333] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.586169 17157 raft_consensus.cc:2243] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.586658 17157 raft_consensus.cc:2272] T 050c5f3e50244a45886a0ee8245165af P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.588726 17157 tablet_replica.cc:333] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.589293 17157 raft_consensus.cc:2243] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.590063 17157 raft_consensus.cc:2272] T a289521cf13242cf80f6cf90c53b8ab4 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.591948 17157 tablet_replica.cc:333] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.592490 17157 raft_consensus.cc:2243] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.592904 17157 raft_consensus.cc:2272] T d729d1bc559949d28af8e08f5a719332 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.594983 17157 tablet_replica.cc:333] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.595762 17157 raft_consensus.cc:2243] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.596765 17157 raft_consensus.cc:2272] T c141b67bfa314acca1c0c3c29d3eaac8 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.597446 26514 consensus_peers.cc:597] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.599135 17157 tablet_replica.cc:333] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.599869 17157 raft_consensus.cc:2243] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.600318 17157 raft_consensus.cc:2272] T 81dffbd4eb274c0595db3619e8eb2058 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.602557 17157 tablet_replica.cc:333] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.603310 17157 raft_consensus.cc:2243] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.603845 17157 raft_consensus.cc:2272] T 269623ffea96490eab12418e6cbd2c01 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.606287 17157 tablet_replica.cc:333] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.606830 17157 raft_consensus.cc:2243] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.607409 17157 raft_consensus.cc:2272] T 113f3dbb8bd5466abbfb043ef9cc4d87 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.609254 17157 tablet_replica.cc:333] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.610038 17157 raft_consensus.cc:2243] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.610651 17157 raft_consensus.cc:2272] T 8fc216f2a251463a9c406ab418b62360 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.613345 17157 tablet_replica.cc:333] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.614215 17157 raft_consensus.cc:2243] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.614809 17157 raft_consensus.cc:2272] T bb49253f88584c74979fb16378c007d1 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.614923 26588 consensus_peers.cc:597] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.616181 26512 consensus_peers.cc:597] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.617367 26514 consensus_peers.cc:597] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.617664 17157 tablet_replica.cc:333] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.618448 17157 raft_consensus.cc:2243] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.619009 17157 raft_consensus.cc:2272] T f76e69771f3c45fd842cbfca4e7b66d5 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.620795 26588 consensus_peers.cc:597] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.621404 17157 tablet_replica.cc:333] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.621986 17157 raft_consensus.cc:2243] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.622771 17157 raft_consensus.cc:2272] T 429d15dc784e4494ad111c92f4812cc6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.624713 17157 tablet_replica.cc:333] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.625384 17157 raft_consensus.cc:2243] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.626223 17157 raft_consensus.cc:2272] T d6eeaddd79764e06b3ac77e3c20ffd49 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.626560 26588 consensus_peers.cc:597] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.628298 17157 tablet_replica.cc:333] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.629179 17157 raft_consensus.cc:2243] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.630133 17157 raft_consensus.cc:2272] T eff188538952450b970105de07be9a85 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.631935 26438 consensus_peers.cc:597] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.632723 17157 tablet_replica.cc:333] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.633313 17157 raft_consensus.cc:2243] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.633737 17157 raft_consensus.cc:2272] T 157df6bf4a63423eb6f1c4f7de58fb9b P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.635598 17157 tablet_replica.cc:333] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.636139 17157 raft_consensus.cc:2243] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.636942 17157 raft_consensus.cc:2272] T 966ed0b8df3447849f8bccb9b1af5b33 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.638749 17157 tablet_replica.cc:333] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.639362 17157 raft_consensus.cc:2243] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.639775 17157 raft_consensus.cc:2272] T a9607fece157456e9d07bf4ef22d370c P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.641698 17157 tablet_replica.cc:333] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.642235 17157 raft_consensus.cc:2243] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.642680 17157 raft_consensus.cc:2272] T 84283d0aa91944d482d29b9d70c80299 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.645201 17157 tablet_replica.cc:333] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.645991 17157 raft_consensus.cc:2243] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.646492 17157 raft_consensus.cc:2272] T 6cc5473d41af4ddc9dd47559842556ca P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.648689 26586 consensus_peers.cc:597] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.649008 17157 tablet_replica.cc:333] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.649649 17157 raft_consensus.cc:2243] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.650075 17157 raft_consensus.cc:2272] T bce90f860c124766af2e9d46c5864016 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.651990 17157 tablet_replica.cc:333] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.652524 17157 raft_consensus.cc:2243] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.652951 17157 raft_consensus.cc:2272] T 948eb78b01d544e6a0e64f60b261edea P bb83296e641848deaddf424cc59fda0e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.654721 17157 tablet_replica.cc:333] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.655277 17157 raft_consensus.cc:2243] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.655702 17157 raft_consensus.cc:2272] T 4aa46be07aa5447cad5a356491417c4c P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.657531 17157 tablet_replica.cc:333] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.658123 17157 raft_consensus.cc:2243] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.658536 17157 raft_consensus.cc:2272] T 4358c31532834ddd8a1e690e2accadfb P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.660369 17157 tablet_replica.cc:333] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.660899 17157 raft_consensus.cc:2243] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.661623 17157 raft_consensus.cc:2272] T ba40df2a50424f46b52f9c4cf6894664 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.663570 17157 tablet_replica.cc:333] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.664104 17157 raft_consensus.cc:2243] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.664489 17157 raft_consensus.cc:2272] T c772c40b55fc4ee88cce69a147fd7ea9 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.666870 17157 tablet_replica.cc:333] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.667654 17157 raft_consensus.cc:2243] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.668679 17157 raft_consensus.cc:2272] T 899048d400bc4540a891c445bcd9bd25 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.670959 26586 consensus_peers.cc:597] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 -> Peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Couldn't send request to peer fc524c91d3c249c0aeb08dae83b98e41. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.671439 17157 tablet_replica.cc:333] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.672053 17157 raft_consensus.cc:2243] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.672950 17157 raft_consensus.cc:2272] T 8f22e34b989c46e1867151870954a0e5 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.674974 17157 tablet_replica.cc:333] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.675570 17157 raft_consensus.cc:2243] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.676056 17157 raft_consensus.cc:2272] T b91ba46bcd3e4802a197552bcbda449f P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.678406 17157 tablet_replica.cc:333] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.679071 17157 raft_consensus.cc:2243] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.680024 17157 raft_consensus.cc:2272] T fe3aff16fd4242f5a9ef1a3038bbe22f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.682461 17157 tablet_replica.cc:333] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.683166 17157 raft_consensus.cc:2243] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.683702 17157 raft_consensus.cc:2272] T 953f66c8075b448fa1ceb677489c5129 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.686297 17157 tablet_replica.cc:333] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.686985 17157 raft_consensus.cc:2243] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.687996 17157 raft_consensus.cc:2272] T 4f9edfeb05dc4d379a4f87f500ab5f26 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.690492 17157 tablet_replica.cc:333] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.691133 17157 raft_consensus.cc:2243] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.691726 17157 raft_consensus.cc:2272] T 32add6893dda4c028a237f05c1f1ec81 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.694442 17157 tablet_replica.cc:333] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.695159 17157 raft_consensus.cc:2243] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.695624 17157 raft_consensus.cc:2272] T 4125b700ff114ebeb4ff2849edf5e291 P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.698506 17157 tablet_replica.cc:333] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.699261 17157 raft_consensus.cc:2243] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.699790 17157 raft_consensus.cc:2272] T 0822222bd071434cb1c7ec59559e02a6 P bb83296e641848deaddf424cc59fda0e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.702051 17157 tablet_replica.cc:333] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.702744 17157 raft_consensus.cc:2243] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.703101 17157 raft_consensus.cc:2272] T df382f80de9947869b3760fc62e30a77 P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.705014 17157 tablet_replica.cc:333] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.705461 17157 raft_consensus.cc:2243] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.706396 17157 raft_consensus.cc:2272] T 7345dd68527a40889384fe8d3710abda P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.709834 17157 tablet_replica.cc:333] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.710696 17157 raft_consensus.cc:2243] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.711404 17157 raft_consensus.cc:2272] T ad43a300a529407993bd1819f2e09aaa P bb83296e641848deaddf424cc59fda0e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.714181 17157 tablet_replica.cc:333] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e: stopping tablet replica
I20260709 12:20:57.714633 17157 raft_consensus.cc:2243] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.715704 17157 raft_consensus.cc:2272] T 759e0f60b4604c2e9abda3833a0e9e6f P bb83296e641848deaddf424cc59fda0e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:57.725272 26588 consensus_peers.cc:597] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.739432 26514 consensus_peers.cc:597] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.754650 26514 consensus_peers.cc:597] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.782276 26514 consensus_peers.cc:597] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.789753 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:20:57.791180 26588 consensus_peers.cc:597] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.826754 26514 consensus_peers.cc:597] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.849596 26514 consensus_peers.cc:597] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.869109 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
W20260709 12:20:57.895223 26514 consensus_peers.cc:597] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.904496 26589 consensus_peers.cc:597] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.931205 26589 consensus_peers.cc:597] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.936715 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:57.937747 17157 tablet_replica.cc:333] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.938745 17157 raft_consensus.cc:2243] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.939375 17157 raft_consensus.cc:2272] T b91ba46bcd3e4802a197552bcbda449f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.941736 17157 tablet_replica.cc:333] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.942620 17157 raft_consensus.cc:2243] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.943688 17157 raft_consensus.cc:2272] T 696254bba321492ab3ca55d7648e313b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.946480 17157 tablet_replica.cc:333] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.947369 17157 raft_consensus.cc:2243] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.947932 17157 raft_consensus.cc:2272] T 095ead81e4364b36a759035a2def0683 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.950547 17157 tablet_replica.cc:333] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.951376 17157 raft_consensus.cc:2243] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.951871 17157 raft_consensus.cc:2272] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.954291 17157 tablet_replica.cc:333] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.955070 17157 raft_consensus.cc:2243] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.955911 17157 raft_consensus.cc:2272] T 235d0175b299499399baaa177b25de35 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.958305 17157 tablet_replica.cc:333] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.959075 17157 raft_consensus.cc:2243] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.960052 17157 raft_consensus.cc:2272] T c6ee0ada749043888b2a134a520a25e1 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.962816 17157 tablet_replica.cc:333] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.963646 17157 raft_consensus.cc:2243] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.964591 17157 raft_consensus.cc:2272] T a97fd66be7ff4a588ce7be8aa1a5b5be P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.967367 17157 tablet_replica.cc:333] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.968127 17157 raft_consensus.cc:2243] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:57.969066 17157 raft_consensus.cc:2272] T 050c5f3e50244a45886a0ee8245165af P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.971709 17157 tablet_replica.cc:333] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.972456 17157 raft_consensus.cc:2243] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.972973 17157 raft_consensus.cc:2272] T d8bad9126db041f7b83ae3e025d0c845 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.975580 17157 tablet_replica.cc:333] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.976346 17157 raft_consensus.cc:2243] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.976804 17157 raft_consensus.cc:2272] T a9f838e26a3f48ebb76d7b98527fe2d0 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.979390 17157 tablet_replica.cc:333] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.980158 17157 raft_consensus.cc:2243] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.980672 17157 raft_consensus.cc:2272] T 3bbbf751e4544950af8ef067e8a1f70f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.983208 17157 tablet_replica.cc:333] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.984004 17157 raft_consensus.cc:2243] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.984509 17157 raft_consensus.cc:2272] T a289521cf13242cf80f6cf90c53b8ab4 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.986873 17157 tablet_replica.cc:333] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.987622 17157 raft_consensus.cc:2243] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.988106 17157 raft_consensus.cc:2272] T c141b67bfa314acca1c0c3c29d3eaac8 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.990644 17157 tablet_replica.cc:333] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.991394 17157 raft_consensus.cc:2243] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.991905 17157 raft_consensus.cc:2272] T d729d1bc559949d28af8e08f5a719332 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.994256 17157 tablet_replica.cc:333] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.994974 17157 raft_consensus.cc:2243] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.995481 17157 raft_consensus.cc:2272] T f94d403c93084635b0e4c70ec3829e1c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:57.997990 17157 tablet_replica.cc:333] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:57.998693 17157 raft_consensus.cc:2243] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:57.999187 17157 raft_consensus.cc:2272] T 7345dd68527a40889384fe8d3710abda P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.001837 17157 tablet_replica.cc:333] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.002585 17157 raft_consensus.cc:2243] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.003558 17157 raft_consensus.cc:2272] T bb49253f88584c74979fb16378c007d1 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.005862 17157 tablet_replica.cc:333] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.006608 17157 raft_consensus.cc:2243] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.007114 17157 raft_consensus.cc:2272] T f76e69771f3c45fd842cbfca4e7b66d5 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.009496 17157 tablet_replica.cc:333] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.010205 17157 raft_consensus.cc:2243] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.010700 17157 raft_consensus.cc:2272] T 429d15dc784e4494ad111c92f4812cc6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.013080 17157 tablet_replica.cc:333] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.013748 17157 raft_consensus.cc:2243] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.014286 17157 raft_consensus.cc:2272] T d6eeaddd79764e06b3ac77e3c20ffd49 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.016860 17157 tablet_replica.cc:333] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.017544 17157 raft_consensus.cc:2243] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.018041 17157 raft_consensus.cc:2272] T 86ba667160b44ddf81a0d48eedb7bdae P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.020664 17157 tablet_replica.cc:333] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.021353 17157 raft_consensus.cc:2243] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.022245 17157 raft_consensus.cc:2272] T 6cc5473d41af4ddc9dd47559842556ca P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.024645 17157 tablet_replica.cc:333] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.025346 17157 raft_consensus.cc:2243] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.025876 17157 raft_consensus.cc:2272] T bce90f860c124766af2e9d46c5864016 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.028285 17157 tablet_replica.cc:333] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.029034 17157 raft_consensus.cc:2243] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.029524 17157 raft_consensus.cc:2272] T 56bca68ee09b4f00a06f68b9fc9ca949 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.031952 17157 tablet_replica.cc:333] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.032723 17157 raft_consensus.cc:2243] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.033280 17157 raft_consensus.cc:2272] T 157df6bf4a63423eb6f1c4f7de58fb9b P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.035758 17157 tablet_replica.cc:333] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.036481 17157 raft_consensus.cc:2243] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.036991 17157 raft_consensus.cc:2272] T 97abcb3c90494ee2b8ad8f45c34923f5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.039456 17157 tablet_replica.cc:333] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
W20260709 12:20:58.039739 26588 consensus_peers.cc:597] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Couldn't send request to peer bb83296e641848deaddf424cc59fda0e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.040138 17157 raft_consensus.cc:2243] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.041080 17157 raft_consensus.cc:2272] T 9f6c90924ba74445ae2edc110c400e78 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.043565 17157 tablet_replica.cc:333] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.044070 17157 raft_consensus.cc:2243] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.044466 17157 raft_consensus.cc:2272] T a9607fece157456e9d07bf4ef22d370c P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.046991 17157 tablet_replica.cc:333] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.047549 17157 raft_consensus.cc:2243] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.048583 17157 raft_consensus.cc:2272] T 84283d0aa91944d482d29b9d70c80299 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.051045 17157 tablet_replica.cc:333] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.051808 17157 raft_consensus.cc:2243] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.052351 17157 raft_consensus.cc:2272] T bf0a016521b542548a73d6110ed04210 P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.052685 26589 consensus_peers.cc:597] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.054405 17157 tablet_replica.cc:333] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.055032 17157 raft_consensus.cc:2243] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.055485 17157 raft_consensus.cc:2272] T b00aa72f3c874a77b2d226ecbebbe834 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.057776 17157 tablet_replica.cc:333] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.058296 17157 raft_consensus.cc:2243] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.059124 17157 raft_consensus.cc:2272] T c772c40b55fc4ee88cce69a147fd7ea9 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.061025 17157 tablet_replica.cc:333] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.061537 17157 raft_consensus.cc:2243] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.061947 17157 raft_consensus.cc:2272] T 85165efa340d47638ef75f49dda75e8a P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.063772 17157 tablet_replica.cc:333] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.064299 17157 raft_consensus.cc:2243] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.065011 17157 raft_consensus.cc:2272] T f97b53680d1349fa8b2bb3c5e067602d P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.066756 17157 tablet_replica.cc:333] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.067301 17157 raft_consensus.cc:2243] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.067667 17157 raft_consensus.cc:2272] T 8f22e34b989c46e1867151870954a0e5 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.069377 17157 tablet_replica.cc:333] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.069856 17157 raft_consensus.cc:2243] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.070307 17157 raft_consensus.cc:2272] T ad43a300a529407993bd1819f2e09aaa P 297900273c7740af9938749b3ce3a80d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.072149 17157 tablet_replica.cc:333] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.072777 17157 raft_consensus.cc:2243] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.073287 17157 raft_consensus.cc:2272] T 52d9fdb0a26d4b43a1940f3ca15c530f P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.075722 17157 tablet_replica.cc:333] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.076365 17157 raft_consensus.cc:2243] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.076870 17157 raft_consensus.cc:2272] T 759e0f60b4604c2e9abda3833a0e9e6f P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.078867 17157 tablet_replica.cc:333] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.079475 17157 raft_consensus.cc:2243] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.080489 17157 raft_consensus.cc:2272] T 953f66c8075b448fa1ceb677489c5129 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.084101 17157 tablet_replica.cc:333] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.084728 17157 raft_consensus.cc:2243] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.085134 17157 raft_consensus.cc:2272] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.086961 17157 tablet_replica.cc:333] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.087484 17157 raft_consensus.cc:2243] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.088198 17157 raft_consensus.cc:2272] T 32add6893dda4c028a237f05c1f1ec81 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.090035 17157 tablet_replica.cc:333] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.090543 17157 raft_consensus.cc:2243] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.091298 17157 raft_consensus.cc:2272] T 0822222bd071434cb1c7ec59559e02a6 P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.093109 17157 tablet_replica.cc:333] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.093566 17157 raft_consensus.cc:2243] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.094300 17157 raft_consensus.cc:2272] T 237bd2b722334bda9aeb29035546ca4c P 297900273c7740af9938749b3ce3a80d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.096258 17157 tablet_replica.cc:333] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d: stopping tablet replica
I20260709 12:20:58.096676 17157 raft_consensus.cc:2243] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.097112 17157 raft_consensus.cc:2272] T 269623ffea96490eab12418e6cbd2c01 P 297900273c7740af9938749b3ce3a80d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.117658 26589 consensus_peers.cc:597] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.136935 26589 consensus_peers.cc:597] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.145629 26589 consensus_peers.cc:597] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.174567 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
W20260709 12:20:58.222956 26589 consensus_peers.cc:597] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:58.227531 26589 consensus_peers.cc:597] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.248417 17157 tablet_server.cc:179] TabletServer@127.16.193.68:0 shutting down...
I20260709 12:20:58.256670 27598 raft_consensus.cc:493] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:58.257238 27598 raft_consensus.cc:515] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
W20260709 12:20:58.258590 26589 consensus_peers.cc:597] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.259439 27598 leader_election.cc:290] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
W20260709 12:20:58.264510 26586 leader_election.cc:336] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111)
W20260709 12:20:58.265286 26589 leader_election.cc:336] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111)
I20260709 12:20:58.265734 26589 leader_election.cc:304] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:58.266376 27598 raft_consensus.cc:2749] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:58.293824 27598 raft_consensus.cc:493] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:58.294389 27598 raft_consensus.cc:515] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "297900273c7740af9938749b3ce3a80d" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 41389 } } peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } }
I20260709 12:20:58.295696 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:58.296571 27598 leader_election.cc:290] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389), fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581)
I20260709 12:20:58.296592 17157 tablet_replica.cc:333] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.298308 17157 raft_consensus.cc:2243] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.298894 17157 raft_consensus.cc:2272] T 063cdd050c3d40fdab471dec902e20e7 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.300946 26586 leader_election.cc:336] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111)
W20260709 12:20:58.301431 26589 leader_election.cc:336] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111)
I20260709 12:20:58.301647 17157 tablet_replica.cc:333] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.301762 26589 leader_election.cc:304] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: 297900273c7740af9938749b3ce3a80d, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:58.302443 27598 raft_consensus.cc:2749] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:20:58.302538 17157 raft_consensus.cc:2243] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.303189 17157 raft_consensus.cc:2272] T 3bbbf751e4544950af8ef067e8a1f70f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.305140 17157 tablet_replica.cc:333] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.305648 17157 raft_consensus.cc:2243] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.306344 17157 raft_consensus.cc:2272] T 095ead81e4364b36a759035a2def0683 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.308318 17157 tablet_replica.cc:333] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.308820 17157 raft_consensus.cc:2243] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.309492 17157 raft_consensus.cc:2272] T 2dcf2bebeaf8451e890bcbe46a6ae967 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.311388 17157 tablet_replica.cc:333] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.312000 17157 raft_consensus.cc:2243] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.312409 17157 raft_consensus.cc:2272] T c6ee0ada749043888b2a134a520a25e1 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.314235 17157 tablet_replica.cc:333] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.314771 17157 raft_consensus.cc:2243] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.315181 17157 raft_consensus.cc:2272] T 696254bba321492ab3ca55d7648e313b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.317054 17157 tablet_replica.cc:333] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.317569 17157 raft_consensus.cc:2243] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.317972 17157 raft_consensus.cc:2272] T 050c5f3e50244a45886a0ee8245165af P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.319840 17157 tablet_replica.cc:333] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.320379 17157 raft_consensus.cc:2243] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.320776 17157 raft_consensus.cc:2272] T d7cc64cd1a8348f68f5b9115f9205e40 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.322578 17157 tablet_replica.cc:333] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.323122 17157 raft_consensus.cc:2243] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.323947 17157 raft_consensus.cc:2272] T a9f838e26a3f48ebb76d7b98527fe2d0 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.326071 17157 tablet_replica.cc:333] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.326699 17157 raft_consensus.cc:2243] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.327597 17157 raft_consensus.cc:2272] T 52d9fdb0a26d4b43a1940f3ca15c530f P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.329531 17157 tablet_replica.cc:333] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.330068 17157 raft_consensus.cc:2243] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.330470 17157 raft_consensus.cc:2272] T 953f66c8075b448fa1ceb677489c5129 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.332290 17157 tablet_replica.cc:333] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.332859 17157 raft_consensus.cc:2243] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.333274 17157 raft_consensus.cc:2272] T 4f9edfeb05dc4d379a4f87f500ab5f26 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.335152 17157 tablet_replica.cc:333] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.335491 27598 raft_consensus.cc:493] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fc524c91d3c249c0aeb08dae83b98e41)
I20260709 12:20:58.335929 17157 raft_consensus.cc:2243] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.336066 27598 raft_consensus.cc:515] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39930cd9790d4943bdec6f9df4ce2641" member_type: VOTER last_known_addr { host: "127.16.193.68" port: 44181 } } peers { permanent_uuid: "fc524c91d3c249c0aeb08dae83b98e41" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41581 } } peers { permanent_uuid: "bb83296e641848deaddf424cc59fda0e" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 37205 } }
I20260709 12:20:58.336556 17157 raft_consensus.cc:2272] T d8bad9126db041f7b83ae3e025d0c845 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.338409 27598 leader_election.cc:290] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581), bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205)
I20260709 12:20:58.339378 17157 tablet_replica.cc:333] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.340242 17157 raft_consensus.cc:2243] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.341102 17157 raft_consensus.cc:2272] T 81dffbd4eb274c0595db3619e8eb2058 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.342769 26586 leader_election.cc:336] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fc524c91d3c249c0aeb08dae83b98e41 (127.16.193.65:41581): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41581: connect: Connection refused (error 111)
W20260709 12:20:58.343382 26588 leader_election.cc:336] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer bb83296e641848deaddf424cc59fda0e (127.16.193.66:37205): Network error: Client connection negotiation failed: client connection to 127.16.193.66:37205: connect: Connection refused (error 111)
I20260709 12:20:58.343690 17157 tablet_replica.cc:333] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.343775 26588 leader_election.cc:304] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39930cd9790d4943bdec6f9df4ce2641; no voters: bb83296e641848deaddf424cc59fda0e, fc524c91d3c249c0aeb08dae83b98e41
I20260709 12:20:58.344452 17157 raft_consensus.cc:2243] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.344512 27598 raft_consensus.cc:2749] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:20:58.345544 17157 raft_consensus.cc:2272] T 86ba667160b44ddf81a0d48eedb7bdae P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.347982 17157 tablet_replica.cc:333] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.348696 17157 raft_consensus.cc:2243] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.349157 17157 raft_consensus.cc:2272] T 113f3dbb8bd5466abbfb043ef9cc4d87 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:58.349815 26589 consensus_peers.cc:597] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 -> Peer 297900273c7740af9938749b3ce3a80d (127.16.193.67:41389): Couldn't send request to peer 297900273c7740af9938749b3ce3a80d. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.67:41389: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:58.351142 17157 tablet_replica.cc:333] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.351802 17157 raft_consensus.cc:2243] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.352231 17157 raft_consensus.cc:2272] T 7345dd68527a40889384fe8d3710abda P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.354154 17157 tablet_replica.cc:333] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.354719 17157 raft_consensus.cc:2243] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.355517 17157 raft_consensus.cc:2272] T f94d403c93084635b0e4c70ec3829e1c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.357751 17157 tablet_replica.cc:333] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.358405 17157 raft_consensus.cc:2243] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.358846 17157 raft_consensus.cc:2272] T eff188538952450b970105de07be9a85 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.361030 17157 tablet_replica.cc:333] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.361552 17157 raft_consensus.cc:2243] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.362285 17157 raft_consensus.cc:2272] T 157df6bf4a63423eb6f1c4f7de58fb9b P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.364050 17157 tablet_replica.cc:333] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.364574 17157 raft_consensus.cc:2243] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.364969 17157 raft_consensus.cc:2272] T 6cc5473d41af4ddc9dd47559842556ca P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.366660 17157 tablet_replica.cc:333] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.367152 17157 raft_consensus.cc:2243] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.367605 17157 raft_consensus.cc:2272] T 56bca68ee09b4f00a06f68b9fc9ca949 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.369735 17157 tablet_replica.cc:333] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.370270 17157 raft_consensus.cc:2243] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.370990 17157 raft_consensus.cc:2272] T 97abcb3c90494ee2b8ad8f45c34923f5 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.373086 17157 tablet_replica.cc:333] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.373714 17157 raft_consensus.cc:2243] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.374424 17157 raft_consensus.cc:2272] T b00aa72f3c874a77b2d226ecbebbe834 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.376272 17157 tablet_replica.cc:333] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.376771 17157 raft_consensus.cc:2243] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.377166 17157 raft_consensus.cc:2272] T bf0a016521b542548a73d6110ed04210 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.378943 17157 tablet_replica.cc:333] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.379547 17157 raft_consensus.cc:2243] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.379947 17157 raft_consensus.cc:2272] T 948eb78b01d544e6a0e64f60b261edea P 39930cd9790d4943bdec6f9df4ce2641 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.381585 17157 tablet_replica.cc:333] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.382087 17157 raft_consensus.cc:2243] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.382498 17157 raft_consensus.cc:2272] T 4aa46be07aa5447cad5a356491417c4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.384486 17157 tablet_replica.cc:333] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.385038 17157 raft_consensus.cc:2243] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.385470 17157 raft_consensus.cc:2272] T 4358c31532834ddd8a1e690e2accadfb P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.387090 17157 tablet_replica.cc:333] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.387614 17157 raft_consensus.cc:2243] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.387995 17157 raft_consensus.cc:2272] T ba40df2a50424f46b52f9c4cf6894664 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.389770 17157 tablet_replica.cc:333] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.390307 17157 raft_consensus.cc:2243] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.391027 17157 raft_consensus.cc:2272] T 8fc216f2a251463a9c406ab418b62360 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.392789 17157 tablet_replica.cc:333] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.393220 17157 raft_consensus.cc:2243] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.393573 17157 raft_consensus.cc:2272] T bb49253f88584c74979fb16378c007d1 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.395087 17157 tablet_replica.cc:333] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.395606 17157 raft_consensus.cc:2243] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.395939 17157 raft_consensus.cc:2272] T 0822222bd071434cb1c7ec59559e02a6 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.397576 17157 tablet_replica.cc:333] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.398001 17157 raft_consensus.cc:2243] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.398322 17157 raft_consensus.cc:2272] T fe3aff16fd4242f5a9ef1a3038bbe22f P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.400053 17157 tablet_replica.cc:333] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.400527 17157 raft_consensus.cc:2243] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.400857 17157 raft_consensus.cc:2272] T 4125b700ff114ebeb4ff2849edf5e291 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.402499 17157 tablet_replica.cc:333] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.402923 17157 raft_consensus.cc:2243] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.403270 17157 raft_consensus.cc:2272] T 9f6c90924ba74445ae2edc110c400e78 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.404922 17157 tablet_replica.cc:333] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.405375 17157 raft_consensus.cc:2243] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.405725 17157 raft_consensus.cc:2272] T 966ed0b8df3447849f8bccb9b1af5b33 P 39930cd9790d4943bdec6f9df4ce2641 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.407368 17157 tablet_replica.cc:333] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.407846 17157 raft_consensus.cc:2243] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.408552 17157 raft_consensus.cc:2272] T a9607fece157456e9d07bf4ef22d370c P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.410255 17157 tablet_replica.cc:333] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.410678 17157 raft_consensus.cc:2243] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.411020 17157 raft_consensus.cc:2272] T 237bd2b722334bda9aeb29035546ca4c P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.412772 17157 tablet_replica.cc:333] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.413178 17157 raft_consensus.cc:2243] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.413511 17157 raft_consensus.cc:2272] T c772c40b55fc4ee88cce69a147fd7ea9 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.415144 17157 tablet_replica.cc:333] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.415655 17157 raft_consensus.cc:2243] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.416328 17157 raft_consensus.cc:2272] T 85165efa340d47638ef75f49dda75e8a P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.418053 17157 tablet_replica.cc:333] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.418471 17157 raft_consensus.cc:2243] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.419095 17157 raft_consensus.cc:2272] T 269623ffea96490eab12418e6cbd2c01 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.421020 17157 tablet_replica.cc:333] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.421516 17157 raft_consensus.cc:2243] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.421911 17157 raft_consensus.cc:2272] T f97b53680d1349fa8b2bb3c5e067602d P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.423676 17157 tablet_replica.cc:333] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.424135 17157 raft_consensus.cc:2243] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:58.424540 17157 raft_consensus.cc:2272] T 899048d400bc4540a891c445bcd9bd25 P 39930cd9790d4943bdec6f9df4ce2641 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.426276 17157 tablet_replica.cc:333] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641: stopping tablet replica
I20260709 12:20:58.426692 17157 raft_consensus.cc:2243] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.427525 17157 raft_consensus.cc:2272] T df382f80de9947869b3760fc62e30a77 P 39930cd9790d4943bdec6f9df4ce2641 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.493348 17157 tablet_server.cc:196] TabletServer@127.16.193.68:0 shutdown complete.
I20260709 12:20:58.552757 17157 master.cc:562] Master@127.16.193.126:33125 shutting down...
I20260709 12:20:58.582890 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:58.583472 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:58.583756 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 718f20e901874dc78478044b5f19e76a: stopping tablet replica
I20260709 12:20:58.607008 17157 master.cc:584] Master@127.16.193.126:33125 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31570 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:20:58.696403 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:35859
I20260709 12:20:58.697511 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:58.703961 27599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.704037 27600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.705710 27602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:58.707122 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:58.708206 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:58.708385 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:58.708496 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599658708485 us; error 0 us; skew 500 ppm
I20260709 12:20:58.708956 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:58.711217 17157 webserver.cc:533] Webserver started at http://127.16.193.126:45945/ using document root <none> and password file <none>
I20260709 12:20:58.711702 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:58.711858 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:58.712061 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:58.713102 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "2e963625c6804f4d90b4e712d636fee2"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-n640"
I20260709 12:20:58.717687 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:58.720904 27607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.721937 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:20:58.722278 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "2e963625c6804f4d90b4e712d636fee2"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-n640"
I20260709 12:20:58.722555 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:58.767472 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:58.768736 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:58.809584 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:35859
I20260709 12:20:58.809767 27658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:35859 every 8 connection(s)
I20260709 12:20:58.813910 27659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.824033 27659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2: Bootstrap starting.
I20260709 12:20:58.828977 27659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:58.833676 27659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2: No bootstrap required, opened a new log
I20260709 12:20:58.835888 27659 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER }
I20260709 12:20:58.836254 27659 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:58.836510 27659 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e963625c6804f4d90b4e712d636fee2, State: Initialized, Role: FOLLOWER
I20260709 12:20:58.837153 27659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER }
I20260709 12:20:58.837612 27659 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:58.837811 27659 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:58.838099 27659 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:58.842715 27659 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER }
I20260709 12:20:58.843271 27659 leader_election.cc:304] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e963625c6804f4d90b4e712d636fee2; no voters: 
I20260709 12:20:58.844373 27659 leader_election.cc:290] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:58.844790 27662 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:58.846112 27662 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 1 LEADER]: Becoming Leader. State: Replica: 2e963625c6804f4d90b4e712d636fee2, State: Running, Role: LEADER
I20260709 12:20:58.846789 27662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER }
I20260709 12:20:58.847239 27659 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:58.852478 27664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e963625c6804f4d90b4e712d636fee2. Latest consensus state: current_term: 1 leader_uuid: "2e963625c6804f4d90b4e712d636fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER } }
I20260709 12:20:58.853428 27664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:58.853434 27663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e963625c6804f4d90b4e712d636fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e963625c6804f4d90b4e712d636fee2" member_type: VOTER } }
I20260709 12:20:58.854991 27663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:58.857961 27669 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:58.866003 27669 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:58.867009 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:58.874567 27669 catalog_manager.cc:1383] Generated new cluster ID: 02cbba93d0674205a17f6956b285ca02
I20260709 12:20:58.874814 27669 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:58.911790 27669 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:58.913096 27669 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:58.926986 27669 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2: Generated new TSK 0
I20260709 12:20:58.927591 27669 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:58.934049 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:58.940234 27680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.940806 27681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:58.942637 27683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:58.943099 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:58.944106 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:58.944336 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:58.944505 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599658944486 us; error 0 us; skew 500 ppm
I20260709 12:20:58.945043 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:58.947324 17157 webserver.cc:533] Webserver started at http://127.16.193.65:41629/ using document root <none> and password file <none>
I20260709 12:20:58.947799 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:58.948016 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:58.948279 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:58.949390 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "fb6b631b99cc4cd299b619ae5936c95e"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-n640"
I20260709 12:20:58.953733 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:58.957142 27688 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.958046 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:20:58.958307 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "fb6b631b99cc4cd299b619ae5936c95e"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-n640"
I20260709 12:20:58.958555 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:58.974435 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:58.975596 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:58.976943 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:58.979130 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:58.979406 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:58.979637 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:58.979827 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.018332 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:41755
I20260709 12:20:59.018433 27750 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:41755 every 8 connection(s)
I20260709 12:20:59.023290 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.031320 27755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.032526 27756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.033885 27758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.035660 17157 server_base.cc:1061] running on GCE node
I20260709 12:20:59.036217 27751 heartbeater.cc:344] Connected to a master server at 127.16.193.126:35859
I20260709 12:20:59.036675 27751 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.037200 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.037513 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.037556 27751 heartbeater.cc:507] Master 127.16.193.126:35859 requested a full tablet report, sending...
I20260709 12:20:59.037776 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599659037756 us; error 0 us; skew 500 ppm
I20260709 12:20:59.038599 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.040151 27624 ts_manager.cc:194] Registered new tserver with Master: fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:20:59.041222 17157 webserver.cc:533] Webserver started at http://127.16.193.66:36953/ using document root <none> and password file <none>
I20260709 12:20:59.041749 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.041951 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.042126 27624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40516
I20260709 12:20:59.042224 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.043819 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "6c16865bd6df4722b0c94e798d94448c"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-n640"
I20260709 12:20:59.048252 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:20:59.051496 27763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.052249 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:20:59.052543 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "6c16865bd6df4722b0c94e798d94448c"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-n640"
I20260709 12:20:59.052834 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.063153 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.064343 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.065696 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.067835 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.068050 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.068291 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.068460 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.106940 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:36455
I20260709 12:20:59.107043 27825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:36455 every 8 connection(s)
I20260709 12:20:59.111881 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:59.119601 27829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:59.120509 27830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.122819 17157 server_base.cc:1061] running on GCE node
W20260709 12:20:59.123481 27833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:59.124565 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:59.124797 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:59.125025 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599659125007 us; error 0 us; skew 500 ppm
I20260709 12:20:59.125562 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:59.126133 27826 heartbeater.cc:344] Connected to a master server at 127.16.193.126:35859
I20260709 12:20:59.126529 27826 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.127352 27826 heartbeater.cc:507] Master 127.16.193.126:35859 requested a full tablet report, sending...
I20260709 12:20:59.128091 17157 webserver.cc:533] Webserver started at http://127.16.193.67:33089/ using document root <none> and password file <none>
I20260709 12:20:59.128607 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:59.128808 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:59.129091 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:59.129437 27624 ts_manager.cc:194] Registered new tserver with Master: 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.130277 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "9382d4e917c248b49f2fc98a184bb5b2"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-n640"
I20260709 12:20:59.130883 27624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40528
I20260709 12:20:59.134883 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:20:59.138224 27837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.138965 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:59.139339 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "9382d4e917c248b49f2fc98a184bb5b2"
format_stamp: "Formatted at 2026-07-09 12:20:59 on dist-test-slave-n640"
I20260709 12:20:59.139626 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:59.162088 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:59.163317 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:59.164745 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:59.166898 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:59.167120 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.167389 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:59.167557 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:59.207409 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:37009
I20260709 12:20:59.207536 27899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:37009 every 8 connection(s)
I20260709 12:20:59.221343 27900 heartbeater.cc:344] Connected to a master server at 127.16.193.126:35859
I20260709 12:20:59.222012 27900 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:59.222767 27900 heartbeater.cc:507] Master 127.16.193.126:35859 requested a full tablet report, sending...
I20260709 12:20:59.224722 27624 ts_manager.cc:194] Registered new tserver with Master: 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.225265 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014697822s
I20260709 12:20:59.226060 27624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40532
I20260709 12:20:59.249795 27624 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40540:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:59.252709 27624 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:59.330529 27864 tablet_service.cc:1511] Processing CreateTablet for tablet 41a21f221f8d4a59b291f754f86ed190 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.331425 27865 tablet_service.cc:1511] Processing CreateTablet for tablet e42dc433a5c54a96acb3f9db9f9ee2d7 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.332091 27864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a21f221f8d4a59b291f754f86ed190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.332854 27865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42dc433a5c54a96acb3f9db9f9ee2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.334378 27863 tablet_service.cc:1511] Processing CreateTablet for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.335737 27863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.337319 27714 tablet_service.cc:1511] Processing CreateTablet for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.338006 27716 tablet_service.cc:1511] Processing CreateTablet for tablet e42dc433a5c54a96acb3f9db9f9ee2d7 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.338757 27714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.340219 27715 tablet_service.cc:1511] Processing CreateTablet for tablet 41a21f221f8d4a59b291f754f86ed190 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.341694 27715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a21f221f8d4a59b291f754f86ed190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.344398 27713 tablet_service.cc:1511] Processing CreateTablet for tablet ef8f6210fb2541a4953756727e78be7d (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.345762 27713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8f6210fb2541a4953756727e78be7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.340319 27862 tablet_service.cc:1511] Processing CreateTablet for tablet ef8f6210fb2541a4953756727e78be7d (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.347599 27716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42dc433a5c54a96acb3f9db9f9ee2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.347139 27862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8f6210fb2541a4953756727e78be7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.352898 27712 tablet_service.cc:1511] Processing CreateTablet for tablet cc54a8122bd449c2b61edc79daaac43b (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.354736 27712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc54a8122bd449c2b61edc79daaac43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.362689 27791 tablet_service.cc:1511] Processing CreateTablet for tablet e42dc433a5c54a96acb3f9db9f9ee2d7 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:59.364200 27791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e42dc433a5c54a96acb3f9db9f9ee2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.368362 27790 tablet_service.cc:1511] Processing CreateTablet for tablet 41a21f221f8d4a59b291f754f86ed190 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:59.369738 27790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41a21f221f8d4a59b291f754f86ed190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.372318 27789 tablet_service.cc:1511] Processing CreateTablet for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:59.373710 27789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea5e6d2c3f84a3bb8f8c3dcab6aa89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.376381 27788 tablet_service.cc:1511] Processing CreateTablet for tablet ef8f6210fb2541a4953756727e78be7d (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:59.377738 27788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef8f6210fb2541a4953756727e78be7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.380492 27787 tablet_service.cc:1511] Processing CreateTablet for tablet cc54a8122bd449c2b61edc79daaac43b (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.382001 27787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc54a8122bd449c2b61edc79daaac43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.384552 27786 tablet_service.cc:1511] Processing CreateTablet for tablet 069d81124dac48beba9a7ad4aae6a254 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.385854 27784 tablet_service.cc:1511] Processing CreateTablet for tablet 7926a3d51fe0495dae7b06a1e7b5e62a (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.385897 27786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069d81124dac48beba9a7ad4aae6a254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.388434 27915 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.387044 27785 tablet_service.cc:1511] Processing CreateTablet for tablet e9f826b0243048c78a2fb584d36c21a2 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.393105 27785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f826b0243048c78a2fb584d36c21a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.394412 27784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7926a3d51fe0495dae7b06a1e7b5e62a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.394824 27865 tablet_service.cc:1511] Processing CreateTablet for tablet cc54a8122bd449c2b61edc79daaac43b (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:59.396253 27865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc54a8122bd449c2b61edc79daaac43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.413142 27714 tablet_service.cc:1511] Processing CreateTablet for tablet 069d81124dac48beba9a7ad4aae6a254 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.416091 27861 tablet_service.cc:1511] Processing CreateTablet for tablet 069d81124dac48beba9a7ad4aae6a254 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:59.422468 27712 tablet_service.cc:1511] Processing CreateTablet for tablet 7926a3d51fe0495dae7b06a1e7b5e62a (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.423195 27861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069d81124dac48beba9a7ad4aae6a254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.423784 27712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7926a3d51fe0495dae7b06a1e7b5e62a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.419783 27860 tablet_service.cc:1511] Processing CreateTablet for tablet e9f826b0243048c78a2fb584d36c21a2 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.423370 27859 tablet_service.cc:1511] Processing CreateTablet for tablet 7926a3d51fe0495dae7b06a1e7b5e62a (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:59.418691 27713 tablet_service.cc:1511] Processing CreateTablet for tablet e9f826b0243048c78a2fb584d36c21a2 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:59.430157 27711 tablet_service.cc:1511] Processing CreateTablet for tablet c16c7cfa0a84454baee93fbe49b9d922 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.439514 27713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f826b0243048c78a2fb584d36c21a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.439723 27860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f826b0243048c78a2fb584d36c21a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.442730 27786 tablet_service.cc:1511] Processing CreateTablet for tablet c16c7cfa0a84454baee93fbe49b9d922 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.443902 27714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069d81124dac48beba9a7ad4aae6a254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.444306 27786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16c7cfa0a84454baee93fbe49b9d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.444043 27858 tablet_service.cc:1511] Processing CreateTablet for tablet c16c7cfa0a84454baee93fbe49b9d922 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:59.445504 27711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16c7cfa0a84454baee93fbe49b9d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.451953 27859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7926a3d51fe0495dae7b06a1e7b5e62a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.454597 27858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16c7cfa0a84454baee93fbe49b9d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.480106 27916 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.481289 27915 tablet_bootstrap.cc:654] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.491730 27917 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.492834 27862 tablet_service.cc:1511] Processing CreateTablet for tablet 563e3f63dffc4ef9a95af236cdb46331 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.494298 27862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563e3f63dffc4ef9a95af236cdb46331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.486845 27715 tablet_service.cc:1511] Processing CreateTablet for tablet 563e3f63dffc4ef9a95af236cdb46331 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.495533 27715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563e3f63dffc4ef9a95af236cdb46331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.498744 27917 tablet_bootstrap.cc:654] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.503790 27784 tablet_service.cc:1511] Processing CreateTablet for tablet 563e3f63dffc4ef9a95af236cdb46331 (DEFAULT_TABLE table=test-workload [id=759a3ee9b42d4619855215ca2f092e6d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:59.505177 27784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563e3f63dffc4ef9a95af236cdb46331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:59.525341 27916 tablet_bootstrap.cc:654] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.541986 27915 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.542610 27915 ts_tablet_manager.cc:1403] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.155s	user 0.015s	sys 0.000s
I20260709 12:20:59.545707 27915 raft_consensus.cc:359] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.546546 27915 raft_consensus.cc:385] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.546916 27915 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.547933 27915 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.549979 27917 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.550586 27917 ts_tablet_manager.cc:1403] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.018s
I20260709 12:20:59.554459 27917 raft_consensus.cc:359] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.555182 27917 raft_consensus.cc:385] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.555512 27917 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.556242 27917 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.559571 27915 ts_tablet_manager.cc:1434] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.005s
I20260709 12:20:59.560336 27826 heartbeater.cc:499] Master 127.16.193.126:35859 was elected leader, sending a full tablet report...
I20260709 12:20:59.560629 27900 heartbeater.cc:499] Master 127.16.193.126:35859 was elected leader, sending a full tablet report...
I20260709 12:20:59.560961 27915 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.567413 27915 tablet_bootstrap.cc:654] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.559672 27917 ts_tablet_manager.cc:1434] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:20:59.569554 27917 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.574020 27915 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.574545 27915 ts_tablet_manager.cc:1403] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:20:59.559645 27916 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.575755 27917 tablet_bootstrap.cc:654] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.576364 27916 ts_tablet_manager.cc:1403] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.097s	user 0.017s	sys 0.002s
I20260709 12:20:59.577600 27915 raft_consensus.cc:359] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.578456 27915 raft_consensus.cc:385] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.578805 27915 raft_consensus.cc:740] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.579686 27915 consensus_queue.cc:260] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.580837 27916 raft_consensus.cc:359] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.581938 27915 ts_tablet_manager.cc:1434] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:59.582432 27916 raft_consensus.cc:385] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.582996 27916 raft_consensus.cc:740] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.583294 27915 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.584497 27916 consensus_queue.cc:260] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.588733 27917 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.589562 27917 ts_tablet_manager.cc:1403] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 12:20:59.590418 27915 tablet_bootstrap.cc:654] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.594635 27916 ts_tablet_manager.cc:1434] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:20:59.595587 27751 heartbeater.cc:499] Master 127.16.193.126:35859 was elected leader, sending a full tablet report...
I20260709 12:20:59.600785 27916 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.606608 27917 raft_consensus.cc:359] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.607616 27917 raft_consensus.cc:385] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.608016 27917 raft_consensus.cc:740] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.608984 27916 tablet_bootstrap.cc:654] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.609104 27922 raft_consensus.cc:493] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.609750 27922 raft_consensus.cc:515] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
W20260709 12:20:59.611279 27827 tablet.cc:2406] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:59.619393 27922 leader_election.cc:290] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.622633 27917 consensus_queue.cc:260] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.634667 27925 raft_consensus.cc:493] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.635331 27925 raft_consensus.cc:515] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.636968 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
W20260709 12:20:59.639271 27767 leader_election.cc:343] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.640241 27917 ts_tablet_manager.cc:1434] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 12:20:59.641570 27917 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.647778 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:20:59.653100 27925 leader_election.cc:290] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
W20260709 12:20:59.649768 27767 leader_election.cc:343] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.664562 27767 leader_election.cc:304] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c; no voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.665712 27922 raft_consensus.cc:2749] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.672291 27915 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.672981 27915 ts_tablet_manager.cc:1403] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.090s	user 0.048s	sys 0.040s
I20260709 12:20:59.674780 27917 tablet_bootstrap.cc:654] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.676059 27915 raft_consensus.cc:359] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.676879 27915 raft_consensus.cc:385] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.677248 27915 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.677685 27916 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.678375 27916 ts_tablet_manager.cc:1403] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.016s
I20260709 12:20:59.678293 27915 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.683084 27916 raft_consensus.cc:359] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.684412 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
W20260709 12:20:59.686094 27692 leader_election.cc:343] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.687047 27916 raft_consensus.cc:385] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.687549 27916 raft_consensus.cc:740] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.687906 27915 ts_tablet_manager.cc:1434] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 12:20:59.687875 27917 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.688851 27917 ts_tablet_manager.cc:1403] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.006s
I20260709 12:20:59.688592 27916 consensus_queue.cc:260] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.692338 27917 raft_consensus.cc:359] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.693197 27917 raft_consensus.cc:385] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.693575 27917 raft_consensus.cc:740] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.694779 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.694591 27917 consensus_queue.cc:260] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.695269 27915 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
W20260709 12:20:59.696754 27689 leader_election.cc:343] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.697244 27689 leader_election.cc:304] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb6b631b99cc4cd299b619ae5936c95e; no voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:20:59.698175 27916 ts_tablet_manager.cc:1434] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.001s
I20260709 12:20:59.698519 27925 raft_consensus.cc:2749] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.700985 27916 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.702759 27915 tablet_bootstrap.cc:654] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.704206 27917 ts_tablet_manager.cc:1434] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:20:59.708124 27917 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.709235 27916 tablet_bootstrap.cc:654] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.711867 27915 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.712406 27915 ts_tablet_manager.cc:1403] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.005s
I20260709 12:20:59.714854 27915 raft_consensus.cc:359] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.715543 27915 raft_consensus.cc:385] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.715847 27915 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.716015 27917 tablet_bootstrap.cc:654] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.716579 27915 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.718740 27915 ts_tablet_manager.cc:1434] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:20:59.719889 27915 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.720870 27921 raft_consensus.cc:493] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.721391 27921 raft_consensus.cc:515] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.722584 27916 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.723320 27916 ts_tablet_manager.cc:1403] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.009s
I20260709 12:20:59.726775 27915 tablet_bootstrap.cc:654] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.726713 27922 raft_consensus.cc:493] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.727509 27922 raft_consensus.cc:515] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.730232 27921 leader_election.cc:290] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:20:59.731494 27922 leader_election.cc:290] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.731907 27917 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.732537 27917 ts_tablet_manager.cc:1403] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 12:20:59.727473 27916 raft_consensus.cc:359] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.733002 27916 raft_consensus.cc:385] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.733088 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:20:59.733410 27916 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.734407 27916 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
W20260709 12:20:59.734790 27767 leader_election.cc:343] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.731639 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
W20260709 12:20:59.737237 27767 leader_election.cc:343] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.737669 27767 leader_election.cc:304] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c; no voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.739413 27922 raft_consensus.cc:2749] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.739316 27917 raft_consensus.cc:359] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.740267 27917 raft_consensus.cc:385] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.740610 27917 raft_consensus.cc:740] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.741803 27916 ts_tablet_manager.cc:1434] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:20:59.745095 27915 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.745687 27915 ts_tablet_manager.cc:1403] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.005s
I20260709 12:20:59.741422 27917 consensus_queue.cc:260] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.748428 27917 ts_tablet_manager.cc:1434] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:20:59.749675 27917 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.752549 27916 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.752117 27915 raft_consensus.cc:359] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.754592 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.756827 27915 raft_consensus.cc:385] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.757143 27915 raft_consensus.cc:740] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
W20260709 12:20:59.757943 27838 leader_election.cc:343] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.758435 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.757948 27915 consensus_queue.cc:260] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
W20260709 12:20:59.760118 27841 leader_election.cc:343] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.760660 27841 leader_election.cc:304] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2; no voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.762017 27921 raft_consensus.cc:2749] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.764060 27916 tablet_bootstrap.cc:654] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.764201 27915 ts_tablet_manager.cc:1434] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:20:59.765420 27915 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.767704 27917 tablet_bootstrap.cc:654] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.769987 27916 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.770579 27916 ts_tablet_manager.cc:1403] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:20:59.772104 27922 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.772670 27922 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.774111 27916 raft_consensus.cc:359] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.775027 27916 raft_consensus.cc:385] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.775085 27922 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.775465 27916 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.776503 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.776263 27916 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.774683 27915 tablet_bootstrap.cc:654] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.777343 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
W20260709 12:20:59.778092 27767 leader_election.cc:343] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:59.779277 27767 leader_election.cc:343] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.779754 27767 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c; no voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.781077 27922 raft_consensus.cc:2749] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:59.782086 27917 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.782680 27917 ts_tablet_manager.cc:1403] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.010s
I20260709 12:20:59.784675 27916 ts_tablet_manager.cc:1434] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:20:59.785066 27915 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.785709 27915 ts_tablet_manager.cc:1403] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.002s
I20260709 12:20:59.785871 27916 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.785948 27917 raft_consensus.cc:359] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.786953 27917 raft_consensus.cc:385] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.787519 27917 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.788482 27917 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.789085 27915 raft_consensus.cc:359] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.789992 27915 raft_consensus.cc:385] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.790356 27915 raft_consensus.cc:740] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.791472 27917 ts_tablet_manager.cc:1434] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:20:59.791430 27915 consensus_queue.cc:260] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.793603 27916 tablet_bootstrap.cc:654] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.794413 27915 ts_tablet_manager.cc:1434] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:59.795168 27917 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.795575 27915 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.796420 27921 raft_consensus.cc:493] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.796967 27921 raft_consensus.cc:515] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.799762 27921 leader_election.cc:290] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.801115 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.801501 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.802057 27801 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:20:59.802248 27726 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:20:59.803712 27841 leader_election.cc:304] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:20:59.804955 27921 raft_consensus.cc:2804] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.805373 27921 raft_consensus.cc:493] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.805743 27921 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.809190 27917 tablet_bootstrap.cc:654] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.810225 27916 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.810878 27916 ts_tablet_manager.cc:1403] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.008s
I20260709 12:20:59.814026 27921 raft_consensus.cc:515] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.815055 27916 raft_consensus.cc:359] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.816244 27916 raft_consensus.cc:385] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.816742 27916 raft_consensus.cc:740] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.818022 27916 consensus_queue.cc:260] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.820827 27916 ts_tablet_manager.cc:1434] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:20:59.820969 27917 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.821624 27917 ts_tablet_manager.cc:1403] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.005s
I20260709 12:20:59.821688 27915 tablet_bootstrap.cc:654] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.822718 27921 leader_election.cc:290] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.823926 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:20:59.824555 27801 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.825697 27917 raft_consensus.cc:359] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.826895 27917 raft_consensus.cc:385] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.827183 27917 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.828083 27917 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.829895 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:20:59.830528 27916 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.830677 27917 ts_tablet_manager.cc:1434] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:20:59.830552 27726 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.832077 27917 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.838057 27916 tablet_bootstrap.cc:654] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.839079 27917 tablet_bootstrap.cc:654] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.840135 27726 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:20:59.841562 27841 leader_election.cc:304] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:20:59.842556 27921 raft_consensus.cc:2804] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:59.842412 27801 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:20:59.853910 27921 raft_consensus.cc:697] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:20:59.855543 27921 consensus_queue.cc:237] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.859117 27915 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.859485 27917 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.859964 27915 ts_tablet_manager.cc:1403] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.003s
I20260709 12:20:59.860405 27917 ts_tablet_manager.cc:1403] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.005s
W20260709 12:20:59.861083 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:59.864029 27916 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:20:59.863869 27915 raft_consensus.cc:359] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.864233 27917 raft_consensus.cc:359] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.864766 27915 raft_consensus.cc:385] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.864704 27916 ts_tablet_manager.cc:1403] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.009s
I20260709 12:20:59.865206 27917 raft_consensus.cc:385] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.865288 27915 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.865764 27917 raft_consensus.cc:740] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.865760 27946 raft_consensus.cc:493] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.866501 27946 raft_consensus.cc:515] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.866528 27915 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.866782 27917 consensus_queue.cc:260] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.868431 27916 raft_consensus.cc:359] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.869426 27916 raft_consensus.cc:385] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.869836 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.871652 27726 raft_consensus.cc:2468] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:20:59.872829 27917 ts_tablet_manager.cc:1434] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:59.873036 27841 leader_election.cc:304] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:20:59.874194 27917 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:20:59.876497 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
W20260709 12:20:59.878631 27838 leader_election.cc:343] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.878962 27921 raft_consensus.cc:2804] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.879493 27921 raft_consensus.cc:493] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.879943 27921 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.882647 27917 tablet_bootstrap.cc:654] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.869859 27916 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.871003 27946 leader_election.cc:290] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.884207 27916 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:20:59.887401 27624 catalog_manager.cc:5697] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:59.888720 27916 ts_tablet_manager.cc:1434] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 12:20:59.898301 27916 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:20:59.904699 27922 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.905228 27922 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.907727 27922 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.908057 27916 tablet_bootstrap.cc:654] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.910251 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.910738 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:20:59.910964 27726 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:20:59.912575 27767 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:20:59.913365 27922 raft_consensus.cc:2804] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.913970 27922 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.914486 27922 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.915302 27949 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.915860 27949 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.916811 27915 ts_tablet_manager.cc:1434] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.056s	user 0.013s	sys 0.021s
I20260709 12:20:59.918318 27915 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:20:59.918843 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
W20260709 12:20:59.920526 27841 leader_election.cc:343] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.922597 27917 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:20:59.924988 27917 ts_tablet_manager.cc:1403] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.051s	user 0.018s	sys 0.011s
I20260709 12:20:59.926913 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.927251 27915 tablet_bootstrap.cc:654] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:59.927724 27801 raft_consensus.cc:2468] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:20:59.929129 27838 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.926383 27922 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.930313 27950 raft_consensus.cc:2804] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.930786 27950 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.931188 27950 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.933070 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:20:59.933722 27726 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.935308 27925 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.935967 27925 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.936625 27949 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.897910 27921 raft_consensus.cc:515] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.899742 27946 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.937762 27946 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.940253 27946 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.940626 27925 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.940825 27875 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:20:59.942212 27726 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:20:59.943570 27767 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:20:59.945252 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:20:59.947196 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.948460 27800 raft_consensus.cc:2393] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9382d4e917c248b49f2fc98a184bb5b2 in current term 1: Already voted for candidate 6c16865bd6df4722b0c94e798d94448c in this term.
I20260709 12:20:59.949167 27917 raft_consensus.cc:359] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.949939 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:20:59.950729 27922 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:20:59.951488 27922 raft_consensus.cc:2804] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:20:59.951915 27692 leader_election.cc:343] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.949954 27917 raft_consensus.cc:385] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.953390 27917 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.950690 27801 raft_consensus.cc:2468] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6b631b99cc4cd299b619ae5936c95e in term 0.
I20260709 12:20:59.955062 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:20:59.955963 27726 raft_consensus.cc:2393] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9382d4e917c248b49f2fc98a184bb5b2 in current term 1: Already voted for candidate 6c16865bd6df4722b0c94e798d94448c in this term.
I20260709 12:20:59.956185 27689 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:20:59.957090 27841 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2; no voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:20:59.957474 27925 raft_consensus.cc:2804] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:59.957921 27925 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.958314 27925 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.958810 27917 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.951987 27922 raft_consensus.cc:697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:20:59.961284 27922 consensus_queue.cc:237] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.967468 27946 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.978384 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:20:59.979138 27726 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.979930 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
W20260709 12:20:59.981598 27838 leader_election.cc:343] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:59.982242 27921 leader_election.cc:290] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:20:59.983727 27925 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.986917 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:20:59.987643 27801 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:59.992080 27915 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:20:59.992909 27915 ts_tablet_manager.cc:1403] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.075s	user 0.026s	sys 0.037s
I20260709 12:20:59.996861 27915 raft_consensus.cc:359] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:20:59.997843 27915 raft_consensus.cc:385] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:59.998268 27915 raft_consensus.cc:740] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:20:59.999375 27915 consensus_queue.cc:260] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.000999 27726 raft_consensus.cc:2468] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.001317 27915 ts_tablet_manager.cc:1434] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:21:00.001750 27916 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:21:00.002420 27916 ts_tablet_manager.cc:1403] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.104s	user 0.046s	sys 0.049s
I20260709 12:21:00.002745 27915 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:21:00.002628 27841 leader_election.cc:304] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:00.005513 27916 raft_consensus.cc:359] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.006618 27916 raft_consensus.cc:385] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.006968 27916 raft_consensus.cc:740] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.007930 27916 consensus_queue.cc:260] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.009699 27921 raft_consensus.cc:2804] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.010464 27921 raft_consensus.cc:697] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
W20260709 12:21:00.012205 27767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:41755, user_credentials={real_user=slave}} blocked reactor thread for 68718.2us
I20260709 12:21:00.011642 27921 consensus_queue.cc:237] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.013923 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:00.015548 27915 tablet_bootstrap.cc:654] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.017098 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
W20260709 12:21:00.018878 27692 leader_election.cc:343] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.019758 27925 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 election: Requested vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.027738 27949 raft_consensus.cc:493] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.028426 27949 raft_consensus.cc:515] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.031512 27917 ts_tablet_manager.cc:1434] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.105s	user 0.011s	sys 0.014s
I20260709 12:21:00.032207 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.032584 27949 leader_election.cc:290] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.032866 27726 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:21:00.034332 27841 leader_election.cc:304] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.034451 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:21:00.034986 27917 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:21:00.035137 27800 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:21:00.035778 27916 ts_tablet_manager.cc:1434] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.033s	user 0.020s	sys 0.012s
I20260709 12:21:00.035215 27949 raft_consensus.cc:2804] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.037171 27949 raft_consensus.cc:493] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.037570 27949 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.039108 27915 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:21:00.039629 27915 ts_tablet_manager.cc:1403] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.005s
I20260709 12:21:00.041260 27917 tablet_bootstrap.cc:654] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.044626 27916 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:00.046247 27915 raft_consensus.cc:359] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.047024 27915 raft_consensus.cc:385] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.047353 27915 raft_consensus.cc:740] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.048102 27915 consensus_queue.cc:260] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.050544 27915 ts_tablet_manager.cc:1434] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:00.051524 27915 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: Bootstrap starting.
I20260709 12:21:00.051978 27916 tablet_bootstrap.cc:654] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.057689 27915 tablet_bootstrap.cc:654] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.059190 27960 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.059752 27960 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.061268 27624 catalog_manager.cc:5697] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.063060 27960 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.063854 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:21:00.065104 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:21:00.065905 27800 raft_consensus.cc:2468] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6b631b99cc4cd299b619ae5936c95e in term 0.
I20260709 12:21:00.067194 27689 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.067518 27917 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:21:00.068087 27917 ts_tablet_manager.cc:1403] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.002s
I20260709 12:21:00.068147 27960 raft_consensus.cc:2804] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.068588 27960 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.068955 27960 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.070504 27915 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: No bootstrap required, opened a new log
I20260709 12:21:00.071171 27915 ts_tablet_manager.cc:1403] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.001s
I20260709 12:21:00.075598 27915 raft_consensus.cc:359] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.076226 27915 raft_consensus.cc:385] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.076498 27915 raft_consensus.cc:740] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.077538 27916 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:21:00.077281 27915 consensus_queue.cc:260] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.070991 27917 raft_consensus.cc:359] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.078155 27916 ts_tablet_manager.cc:1403] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.009s
I20260709 12:21:00.078454 27917 raft_consensus.cc:385] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.078794 27917 raft_consensus.cc:740] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.079641 27915 ts_tablet_manager.cc:1434] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:21:00.005390 27946 raft_consensus.cc:2749] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.080366 27960 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.087014 27960 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 election: Requested vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.047097 27949 raft_consensus.cc:515] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.080966 27917 consensus_queue.cc:260] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.087383 27620 catalog_manager.cc:5697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 1 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.089818 27949 leader_election.cc:290] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.081043 27916 raft_consensus.cc:359] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.091027 27873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:00.084395 27950 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.091958 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:00.083540 27801 raft_consensus.cc:2468] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6b631b99cc4cd299b619ae5936c95e in term 1.
I20260709 12:21:00.092614 27799 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.093453 27950 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.093674 27874 raft_consensus.cc:2393] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb6b631b99cc4cd299b619ae5936c95e in current term 1: Already voted for candidate 9382d4e917c248b49f2fc98a184bb5b2 in this term.
I20260709 12:21:00.094406 27725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.091060 27916 raft_consensus.cc:385] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.095266 27725 raft_consensus.cc:2393] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9382d4e917c248b49f2fc98a184bb5b2 in current term 1: Already voted for candidate fb6b631b99cc4cd299b619ae5936c95e in this term.
I20260709 12:21:00.095327 27916 raft_consensus.cc:740] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.090545 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.096657 27916 consensus_queue.cc:260] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.081390 27922 raft_consensus.cc:493] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.095811 27689 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:00.097622 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:00.097038 27726 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.097865 27922 raft_consensus.cc:515] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:00.098213 27960 raft_consensus.cc:2804] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.098985 27960 raft_consensus.cc:697] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 LEADER]: Becoming Leader. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Running, Role: LEADER
I20260709 12:21:00.099911 27960 consensus_queue.cc:237] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.100747 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:21:00.101401 27874 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:21:00.098317 27801 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.102643 27767 leader_election.cc:304] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: 
I20260709 12:21:00.104259 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "fb6b631b99cc4cd299b619ae5936c95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:00.104568 27922 leader_election.cc:290] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009), fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:21:00.105341 27922 raft_consensus.cc:2804] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.105422 27950 raft_consensus.cc:493] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.105710 27922 raft_consensus.cc:493] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.105911 27689 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb6b631b99cc4cd299b619ae5936c95e; no voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:00.106197 27922 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.106017 27950 raft_consensus.cc:515] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.107259 27925 raft_consensus.cc:2749] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.108434 27950 leader_election.cc:290] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.108989 27724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
W20260709 12:21:00.110435 27841 leader_election.cc:343] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:00.110610 27725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.096009 27917 ts_tablet_manager.cc:1434] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:21:00.111754 27725 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:21:00.112046 27917 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: Bootstrap starting.
I20260709 12:21:00.112556 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:21:00.113418 27800 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:21:00.114905 27838 leader_election.cc:304] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:00.116091 27950 raft_consensus.cc:2804] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.116562 27950 raft_consensus.cc:493] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.116988 27950 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.118029 27917 tablet_bootstrap.cc:654] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.104580 27916 ts_tablet_manager.cc:1434] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:21:00.120265 27916 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:00.125913 27726 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.127178 27841 leader_election.cc:304] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.128047 27949 raft_consensus.cc:2804] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.128571 27949 raft_consensus.cc:697] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:00.128748 27801 raft_consensus.cc:2468] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.129432 27949 consensus_queue.cc:237] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.130460 27838 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:00.129961 27916 tablet_bootstrap.cc:654] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.131601 27946 raft_consensus.cc:2804] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.131753 27921 raft_consensus.cc:493] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.132238 27921 raft_consensus.cc:515] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.134789 27921 leader_election.cc:290] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.135977 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:21:00.136637 27800 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:21:00.136713 27725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.137526 27725 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 0.
I20260709 12:21:00.137950 27838 leader_election.cc:304] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: 
I20260709 12:21:00.139489 27875 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:00.142877 27921 raft_consensus.cc:2804] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.143458 27921 raft_consensus.cc:493] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.143894 27921 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.145151 27916 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:21:00.145709 27916 ts_tablet_manager.cc:1403] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.005s
I20260709 12:21:00.127712 27922 raft_consensus.cc:515] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:00.152772 27916 raft_consensus.cc:359] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.153515 27916 raft_consensus.cc:385] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.138934 27799 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.153784 27916 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.154006 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:00.154735 27874 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.156884 27950 raft_consensus.cc:515] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.159751 27950 leader_election.cc:290] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.156821 27916 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.161106 27922 leader_election.cc:290] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Requested vote from peers 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009), fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:21:00.162348 27725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.162956 27725 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.162971 27916 ts_tablet_manager.cc:1434] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:21:00.166494 27916 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:00.166383 27801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:00.167053 27801 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.172221 27949 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.172978 27949 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.176209 27946 raft_consensus.cc:697] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:00.176724 27724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.177155 27946 consensus_queue.cc:237] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.179834 27916 tablet_bootstrap.cc:654] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:00.180326 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
I20260709 12:21:00.153906 27921 raft_consensus.cc:515] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.181165 27799 raft_consensus.cc:2393] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9382d4e917c248b49f2fc98a184bb5b2 in current term 1: Already voted for candidate fb6b631b99cc4cd299b619ae5936c95e in this term.
I20260709 12:21:00.182276 27838 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2; no voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:00.183363 27724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.184157 27724 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.184166 27949 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.183455 27950 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.191231 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.193359 27726 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.192349 27725 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:00.196198 27767 leader_election.cc:304] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.195643 27623 catalog_manager.cc:5697] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e reported cstate change: term changed from 0 to 1, leader changed from <none> to fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65). New cstate: current_term: 1 leader_uuid: "fb6b631b99cc4cd299b619ae5936c95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.197266 27922 raft_consensus.cc:2804] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.198797 27922 raft_consensus.cc:697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:00.200212 27922 consensus_queue.cc:237] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:00.179630 27874 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:00.222432 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:00.223097 27799 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.227771 27724 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.225750 27801 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.233006 27841 leader_election.cc:304] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.233811 27726 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.236174 27622 catalog_manager.cc:5697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 1 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.239830 27838 leader_election.cc:304] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: 
I20260709 12:21:00.240834 27949 raft_consensus.cc:2804] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.241783 27968 raft_consensus.cc:2804] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.245532 27623 catalog_manager.cc:5697] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.246927 27623 catalog_manager.cc:5697] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.228415 27950 raft_consensus.cc:2749] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.226990 27921 leader_election.cc:290] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:00.255352 27799 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 1.
I20260709 12:21:00.259409 27968 raft_consensus.cc:697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:00.257627 27917 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: No bootstrap required, opened a new log
I20260709 12:21:00.257503 27949 raft_consensus.cc:697] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:00.260787 27949 consensus_queue.cc:237] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.260463 27968 consensus_queue.cc:237] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.262012 27916 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: No bootstrap required, opened a new log
I20260709 12:21:00.262542 27916 ts_tablet_manager.cc:1403] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.096s	user 0.034s	sys 0.029s
I20260709 12:21:00.265620 27916 raft_consensus.cc:359] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.266413 27916 raft_consensus.cc:385] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.266804 27916 raft_consensus.cc:740] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.266429 27917 ts_tablet_manager.cc:1403] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: Time spent bootstrapping tablet: real 0.155s	user 0.022s	sys 0.064s
I20260709 12:21:00.267735 27916 consensus_queue.cc:260] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.273910 27916 ts_tablet_manager.cc:1434] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:00.276381 27917 raft_consensus.cc:359] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.277082 27917 raft_consensus.cc:385] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:00.277272 27917 raft_consensus.cc:740] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Initialized, Role: FOLLOWER
I20260709 12:21:00.277926 27917 consensus_queue.cc:260] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.279937 27917 ts_tablet_manager.cc:1434] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:00.281030 27622 catalog_manager.cc:5697] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.282859 27622 catalog_manager.cc:5697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.355443 27968 consensus_queue.cc:1048] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.367929 27968 consensus_queue.cc:1048] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.388559 27922 raft_consensus.cc:493] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.389019 27922 raft_consensus.cc:515] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.390820 27922 leader_election.cc:290] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.391671 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.391934 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:21:00.392403 27726 raft_consensus.cc:2393] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c16865bd6df4722b0c94e798d94448c in current term 1: Already voted for candidate 9382d4e917c248b49f2fc98a184bb5b2 in this term.
I20260709 12:21:00.393576 27767 leader_election.cc:304] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c; no voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:00.394308 27922 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.399111 27922 raft_consensus.cc:2749] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:00.421950 27958 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.425525 27958 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.439850 27957 raft_consensus.cc:493] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:00.440407 27957 raft_consensus.cc:515] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.442420 27957 leader_election.cc:290] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.445204 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e" is_pre_election: true
I20260709 12:21:00.445550 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:21:00.445925 27726 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:21:00.446295 27874 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 0.
I20260709 12:21:00.447052 27767 leader_election.cc:304] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.447923 27922 raft_consensus.cc:2804] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:00.448390 27922 raft_consensus.cc:493] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:00.448683 27922 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.455813 27922 raft_consensus.cc:515] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.457836 27922 leader_election.cc:290] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:00.458621 27726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:00.458732 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:00.459393 27726 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.459493 27874 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:00.464469 27726 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:00.464986 27874 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:00.465626 27767 leader_election.cc:304] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:00.466545 27922 raft_consensus.cc:2804] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:00.467422 27922 raft_consensus.cc:697] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:00.468924 27922 consensus_queue.cc:237] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:00.469391 27968 consensus_queue.cc:1048] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.480005 27624 catalog_manager.cc:5697] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 1 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:00.481041 27968 consensus_queue.cc:1048] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.561317 27960 consensus_queue.cc:1048] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.573304 27990 consensus_queue.cc:1048] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.581188 17157 ts_manager.cc:295] Set tserver state for fb6b631b99cc4cd299b619ae5936c95e to MAINTENANCE_MODE
I20260709 12:21:00.582146 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
I20260709 12:21:00.616415 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:00.617249 17157 tablet_replica.cc:333] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.617934 17157 raft_consensus.cc:2243] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.618283 17157 raft_consensus.cc:2272] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.620373 17157 tablet_replica.cc:333] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.620906 17157 raft_consensus.cc:2243] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.621225 17157 raft_consensus.cc:2272] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.623056 17157 tablet_replica.cc:333] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.623646 17157 raft_consensus.cc:2243] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.624047 17157 raft_consensus.cc:2272] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.625706 17157 tablet_replica.cc:333] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.626248 17157 raft_consensus.cc:2243] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.626600 17157 raft_consensus.cc:2272] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.628270 17157 tablet_replica.cc:333] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.628880 17157 raft_consensus.cc:2243] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.629308 17157 raft_consensus.cc:2272] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.631099 17157 tablet_replica.cc:333] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.631356 27922 consensus_queue.cc:1048] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.631652 17157 raft_consensus.cc:2243] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.632206 17157 raft_consensus.cc:2272] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.634733 17157 tablet_replica.cc:333] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.635383 17157 raft_consensus.cc:2243] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.635968 17157 raft_consensus.cc:2272] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.639108 17157 tablet_replica.cc:333] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.639822 17157 raft_consensus.cc:2243] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:00.640846 17157 raft_consensus.cc:2272] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:00.644506 27767 consensus_peers.cc:597] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.647109 17157 tablet_replica.cc:333] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.647921 17157 raft_consensus.cc:2243] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.648445 17157 raft_consensus.cc:2272] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.651284 17157 tablet_replica.cc:333] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: stopping tablet replica
I20260709 12:21:00.651921 17157 raft_consensus.cc:2243] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.652460 17157 raft_consensus.cc:2272] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:00.655943 27841 consensus_peers.cc:597] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.661221 27841 consensus_peers.cc:597] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.681063 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
I20260709 12:21:00.720160 27949 consensus_queue.cc:1048] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:00.720484 27968 consensus_queue.cc:1048] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:00.737911 27968 consensus_queue.cc:1048] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:00.750295 27841 consensus_peers.cc:597] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.822829 27767 consensus_peers.cc:597] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:00.854504 27968 consensus_queue.cc:1048] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:00.862171 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:00.868949 27841 consensus_peers.cc:597] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.892206 27767 consensus_peers.cc:597] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:00.905063 27841 consensus_peers.cc:597] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:01.073916 27957 consensus_queue.cc:1048] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:01.080065 27841 consensus_peers.cc:597] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:01.863406 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:02.071023 28004 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb6b631b99cc4cd299b619ae5936c95e)
I20260709 12:21:02.071659 28004 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:02.073767 28004 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:02.074769 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c16865bd6df4722b0c94e798d94448c" is_pre_election: true
W20260709 12:21:02.077405 27841 leader_election.cc:336] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111)
I20260709 12:21:02.077786 27841 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2; no voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:02.078368 28004 raft_consensus.cc:2749] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:02.078990 27958 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb6b631b99cc4cd299b619ae5936c95e)
I20260709 12:21:02.079476 27958 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:02.081374 27958 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:02.082612 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2" is_pre_election: true
I20260709 12:21:02.083220 27874 raft_consensus.cc:2468] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 1.
I20260709 12:21:02.084911 27767 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: 
I20260709 12:21:02.085899 28006 raft_consensus.cc:2804] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:02.085983 27767 leader_election.cc:336] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111)
I20260709 12:21:02.086293 28006 raft_consensus.cc:493] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb6b631b99cc4cd299b619ae5936c95e)
I20260709 12:21:02.086678 28006 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:02.092597 28006 raft_consensus.cc:515] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:02.094306 28006 leader_election.cc:290] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:02.095742 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c16c7cfa0a84454baee93fbe49b9d922" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:02.096470 27874 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:02.097868 27767 leader_election.cc:336] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111)
I20260709 12:21:02.101994 27874 raft_consensus.cc:2468] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:02.102933 27767 leader_election.cc:304] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: fb6b631b99cc4cd299b619ae5936c95e
I20260709 12:21:02.103662 28006 raft_consensus.cc:2804] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:02.104086 28006 raft_consensus.cc:697] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:02.104801 28006 consensus_queue.cc:237] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:02.111896 27624 catalog_manager.cc:5697] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:02.230211 27841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111) [suppressed 124 similar messages]
W20260709 12:21:02.560683 27767 consensus_peers.cc:597] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.704252 27874 raft_consensus.cc:1275] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:02.705492 27958 consensus_queue.cc:1048] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:02.859947 27767 consensus_peers.cc:597] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:02.864588 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:02.944336 27841 consensus_peers.cc:597] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.160948 27841 consensus_peers.cc:597] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.191347 27841 consensus_peers.cc:597] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.374733 27841 consensus_peers.cc:597] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.414633 27767 consensus_peers.cc:597] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.523597 27767 consensus_peers.cc:597] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.548132 27841 consensus_peers.cc:597] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.587271 27841 consensus_peers.cc:597] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:03.865948 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:04.866989 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:05.030344 27767 consensus_peers.cc:597] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:05.306632 27767 consensus_peers.cc:597] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.438812 27841 consensus_peers.cc:597] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.556102 27841 consensus_peers.cc:597] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:05.699990 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:05.707967 28019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:05.708451 28020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:05.710656 28022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:05.711318 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:05.712926 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:05.713169 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:05.713364 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599665713349 us; error 0 us; skew 500 ppm
I20260709 12:21:05.713958 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:05.716755 17157 webserver.cc:533] Webserver started at http://127.16.193.65:41629/ using document root <none> and password file <none>
I20260709 12:21:05.717311 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:05.717514 17157 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:21:05.721838 17157 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:21:05.725988 28027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:05.726955 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:05.727222 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "fb6b631b99cc4cd299b619ae5936c95e"
format_stamp: "Formatted at 2026-07-09 12:20:58 on dist-test-slave-n640"
I20260709 12:21:05.727646 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:05.741945 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:05.743201 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:05.744807 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:05.748947 28034 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:21:05.763383 27841 consensus_peers.cc:597] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:05.802726 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:21:05.803121 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.055s	user 0.003s	sys 0.000s
I20260709 12:21:05.803534 17157 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:21:05.809023 28034 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
W20260709 12:21:05.815073 27767 consensus_peers.cc:597] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:05.824976 28034 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.825773 28034 tablet_bootstrap.cc:492] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.826380 28034 ts_tablet_manager.cc:1403] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:21:05.828676 28034 raft_consensus.cc:359] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.829738 28034 raft_consensus.cc:740] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.830402 28034 consensus_queue.cc:260] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.833312 28034 ts_tablet_manager.cc:1434] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:21:05.834126 28034 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.845978 17157 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:21:05.846328 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.043s	user 0.040s	sys 0.000s
I20260709 12:21:05.847457 28034 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.848263 28034 tablet_bootstrap.cc:492] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.848935 28034 ts_tablet_manager.cc:1403] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.008s
I20260709 12:21:05.851082 28034 raft_consensus.cc:359] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.851677 28034 raft_consensus.cc:740] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.852293 28034 consensus_queue.cc:260] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.853900 28034 ts_tablet_manager.cc:1434] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 12:21:05.854686 28034 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
W20260709 12:21:05.862336 27841 consensus_peers.cc:597] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:05.868003 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:05.868541 28034 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.869298 28034 tablet_bootstrap.cc:492] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.869910 28034 ts_tablet_manager.cc:1403] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.005s
I20260709 12:21:05.872318 28034 raft_consensus.cc:359] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.872862 28034 raft_consensus.cc:740] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.873550 28034 consensus_queue.cc:260] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.875305 28034 ts_tablet_manager.cc:1434] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:21:05.876188 28034 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.891767 28034 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.892432 28034 tablet_bootstrap.cc:492] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.892983 28034 ts_tablet_manager.cc:1403] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:21:05.895465 28034 raft_consensus.cc:359] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.895980 28034 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.896621 28034 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.898133 28034 ts_tablet_manager.cc:1434] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:05.898972 28034 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.909493 28034 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.910202 28034 tablet_bootstrap.cc:492] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.910750 28034 ts_tablet_manager.cc:1403] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 12:21:05.912936 28034 raft_consensus.cc:359] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.913614 28034 raft_consensus.cc:740] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.914438 28034 consensus_queue.cc:260] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.915614 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:41755
I20260709 12:21:05.915757 28097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:41755 every 8 connection(s)
I20260709 12:21:05.916790 28034 ts_tablet_manager.cc:1434] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:21:05.917570 28034 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.923547 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:05.941612 28034 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.942701 28034 tablet_bootstrap.cc:492] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.943444 28034 ts_tablet_manager.cc:1403] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.011s
W20260709 12:21:05.947125 27767 consensus_peers.cc:597] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:05.949285 28034 raft_consensus.cc:359] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.950142 28034 raft_consensus.cc:740] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.950945 28034 consensus_queue.cc:260] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.953112 28034 ts_tablet_manager.cc:1434] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:05.954010 28034 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.956096 28099 heartbeater.cc:344] Connected to a master server at 127.16.193.126:35859
I20260709 12:21:05.956540 28099 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:05.957409 28099 heartbeater.cc:507] Master 127.16.193.126:35859 requested a full tablet report, sending...
I20260709 12:21:05.961305 27874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:05.962088 27874 raft_consensus.cc:606] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:05.965140 27874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:05.965864 27874 raft_consensus.cc:606] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:05.966989 27624 ts_manager.cc:194] Re-registered known tserver with Master: fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:21:05.967093 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:05.967870 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:05.968591 28034 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.968657 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:05.969564 28034 tablet_bootstrap.cc:492] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.970254 28034 ts_tablet_manager.cc:1403] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:21:05.972982 28034 raft_consensus.cc:359] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.973659 28034 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.974382 28034 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.975169 27624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44364
I20260709 12:21:05.976455 28034 ts_tablet_manager.cc:1434] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:21:05.977344 28034 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:05.978387 28099 heartbeater.cc:499] Master 127.16.193.126:35859 was elected leader, sending a full tablet report...
I20260709 12:21:05.990459 28034 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.991241 28034 tablet_bootstrap.cc:492] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:05.991845 28034 ts_tablet_manager.cc:1403] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:21:05.993777 28034 raft_consensus.cc:359] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.994220 28034 raft_consensus.cc:740] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:05.994685 28034 consensus_queue.cc:260] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:05.996587 28034 ts_tablet_manager.cc:1434] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:05.997339 28034 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:06.009711 28034 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.010444 28034 tablet_bootstrap.cc:492] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:06.010970 28034 ts_tablet_manager.cc:1403] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:21:06.012807 28034 raft_consensus.cc:359] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.013269 28034 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.013827 28034 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.015538 28034 ts_tablet_manager.cc:1434] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.004s	user 0.008s	sys 0.000s
I20260709 12:21:06.016301 28034 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap starting.
I20260709 12:21:06.026486 28034 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.027115 28034 tablet_bootstrap.cc:492] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Bootstrap complete.
I20260709 12:21:06.027685 28034 ts_tablet_manager.cc:1403] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.002s
I20260709 12:21:06.029551 28034 raft_consensus.cc:359] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:06.030243 28034 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6b631b99cc4cd299b619ae5936c95e, State: Initialized, Role: FOLLOWER
I20260709 12:21:06.030913 28034 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:06.032692 28034 ts_tablet_manager.cc:1434] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 12:21:06.176254 28062 raft_consensus.cc:3060] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.295789 28017 raft_consensus.cc:993] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:06.296257 28017 raft_consensus.cc:1081] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:06.297703 27799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:06.298255 27799 raft_consensus.cc:493] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.298584 27799 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.305714 27799 raft_consensus.cc:515] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.307664 27799 leader_election.cc:290] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:06.308320 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:06.308794 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:06.309173 28071 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.309513 27874 raft_consensus.cc:3055] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.309841 27874 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:06.310871 27874 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.312417 27874 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.317973 28071 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:06.319281 27767 leader_election.cc:304] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:06.320036 27958 raft_consensus.cc:2804] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.320514 27874 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:06.321101 27958 raft_consensus.cc:697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:06.322299 27958 consensus_queue.cc:237] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.330346 27624 catalog_manager.cc:5697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.376312 28069 raft_consensus.cc:993] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:06.376819 28017 raft_consensus.cc:1081] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:06.378260 27799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:06.378783 27799 raft_consensus.cc:493] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.379127 27799 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.384785 27799 raft_consensus.cc:515] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.386826 27799 leader_election.cc:290] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:06.387460 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:06.387966 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f826b0243048c78a2fb584d36c21a2" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:06.388110 28071 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.388665 27874 raft_consensus.cc:3055] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.388978 27874 raft_consensus.cc:740] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:06.389842 27874 consensus_queue.cc:260] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.391125 27874 raft_consensus.cc:3060] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.393544 28071 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:06.394585 27767 leader_election.cc:304] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:06.395359 27958 raft_consensus.cc:2804] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.395774 27958 raft_consensus.cc:697] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:06.396634 27958 consensus_queue.cc:237] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:06.397073 27874 raft_consensus.cc:2468] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:06.404670 27624 catalog_manager.cc:5697] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:06.869130 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:06.869669 27874 raft_consensus.cc:1275] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.870939 27958 consensus_queue.cc:1048] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.880797 28071 raft_consensus.cc:1275] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.882386 28125 consensus_queue.cc:1048] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:06.979687 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.996315 27799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:06.996935 27799 raft_consensus.cc:606] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:06.999611 27799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:07.000089 27799 raft_consensus.cc:606] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:07.002394 27874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:07.002866 27874 raft_consensus.cc:606] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:07.004209 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:07.004722 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:07.004776 27874 raft_consensus.cc:1275] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.005373 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:07.006137 27958 consensus_queue.cc:1048] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.014482 28071 raft_consensus.cc:1275] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.016733 28125 consensus_queue.cc:1048] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.308156 28125 raft_consensus.cc:993] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:07.308630 27958 raft_consensus.cc:1081] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:07.310144 27874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:07.310597 27874 raft_consensus.cc:493] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.310870 27874 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.316606 27874 raft_consensus.cc:515] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.318276 27874 leader_election.cc:290] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:07.319181 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:07.319497 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563e3f63dffc4ef9a95af236cdb46331" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:07.319842 28062 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.320256 27799 raft_consensus.cc:3055] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.320601 27799 raft_consensus.cc:740] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:07.321462 27799 consensus_queue.cc:260] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.322597 27799 raft_consensus.cc:3060] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.325641 28062 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:07.326721 27841 leader_election.cc:304] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:07.327587 28124 raft_consensus.cc:2804] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.328646 28124 raft_consensus.cc:697] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:07.329280 27799 raft_consensus.cc:2468] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:07.330212 28124 consensus_queue.cc:237] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.338253 27622 catalog_manager.cc:5697] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 1 to 2, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:07.391573 28135 raft_consensus.cc:993] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:07.392139 28124 raft_consensus.cc:1081] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:07.393846 27799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:07.394366 27799 raft_consensus.cc:493] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.394691 27799 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.399369 27799 raft_consensus.cc:515] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.400841 27799 leader_election.cc:290] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:07.401487 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:07.401845 27874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:07.402195 28062 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.402459 27874 raft_consensus.cc:3055] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.402809 27874 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:07.403539 27874 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.404526 27874 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.407840 28062 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:07.408927 27767 leader_election.cc:304] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:07.409535 27958 raft_consensus.cc:2804] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.409951 27958 raft_consensus.cc:697] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:07.410955 27874 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:07.410818 27958 consensus_queue.cc:237] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.417424 27622 catalog_manager.cc:5697] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.499481 28116 raft_consensus.cc:993] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:07.499977 27958 raft_consensus.cc:1081] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:07.501340 27874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:07.501720 27874 raft_consensus.cc:493] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.501957 27874 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.506405 27874 raft_consensus.cc:515] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.507838 27874 leader_election.cc:290] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 3 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:07.508678 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:07.508888 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7926a3d51fe0495dae7b06a1e7b5e62a" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:07.509320 28062 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.509387 27799 raft_consensus.cc:3055] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:07.509795 27799 raft_consensus.cc:740] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:07.510532 27799 consensus_queue.cc:260] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.511725 27799 raft_consensus.cc:3060] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.516579 28062 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 3.
I20260709 12:21:07.517747 27841 leader_election.cc:304] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:07.518283 27799 raft_consensus.cc:2468] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 3.
I20260709 12:21:07.518412 28124 raft_consensus.cc:2804] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:07.519008 28124 raft_consensus.cc:697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:07.519871 28124 consensus_queue.cc:237] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:07.528798 27622 catalog_manager.cc:5697] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 2 to 3, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:07.813616 28062 raft_consensus.cc:1275] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.814888 28124 consensus_queue.cc:1048] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.825673 27799 raft_consensus.cc:1275] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.827378 28124 consensus_queue.cc:1048] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:07.870218 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:07.971653 27874 raft_consensus.cc:1275] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.973119 27958 consensus_queue.cc:1048] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.983470 28062 raft_consensus.cc:1275] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.985257 28148 consensus_queue.cc:1048] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:08.014683 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:08.022269 27874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:08.022804 27874 raft_consensus.cc:606] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:08.025490 27874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:08.025943 27874 raft_consensus.cc:606] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:08.028182 27799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:08.028728 27799 raft_consensus.cc:606] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:08.029505 28062 raft_consensus.cc:1275] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:08.030051 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:08.030748 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:08.031136 28124 consensus_queue.cc:1048] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:08.031404 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:08.043551 27799 raft_consensus.cc:1275] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:08.045348 28124 consensus_queue.cc:1048] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:08.244715 28135 raft_consensus.cc:993] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:08.244817 28116 raft_consensus.cc:993] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:08.245242 28124 raft_consensus.cc:1081] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:08.245440 28148 raft_consensus.cc:1081] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:08.246830 27799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:08.247044 27874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:08.247421 27799 raft_consensus.cc:493] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.247594 27874 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.247814 27799 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.247967 27874 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.254550 27874 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.254551 27799 raft_consensus.cc:515] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.256475 27799 leader_election.cc:290] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:08.256484 27874 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:08.257644 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:08.257805 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:08.258316 28062 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.258340 27875 raft_consensus.cc:3055] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.258852 27875 raft_consensus.cc:740] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:08.259120 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069d81124dac48beba9a7ad4aae6a254" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:08.259766 28071 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.259701 27875 consensus_queue.cc:260] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.260993 27875 raft_consensus.cc:3060] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.263574 27799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:08.264184 27799 raft_consensus.cc:3055] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.264482 27799 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:08.265368 27799 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.266605 27799 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.272258 28071 raft_consensus.cc:2468] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:08.272405 27875 raft_consensus.cc:2468] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:08.273321 27799 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:08.273592 27767 leader_election.cc:304] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:08.274459 28148 raft_consensus.cc:2804] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.274495 27838 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, 9382d4e917c248b49f2fc98a184bb5b2; no voters: 
I20260709 12:21:08.275075 28148 raft_consensus.cc:697] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:08.275553 28124 raft_consensus.cc:2804] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.276104 28124 raft_consensus.cc:697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:08.276062 28148 consensus_queue.cc:237] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.277171 28124 consensus_queue.cc:237] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.277436 28062 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:08.285362 27622 catalog_manager.cc:5697] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.287077 27624 catalog_manager.cc:5697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 1 to 2, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.385360 28151 raft_consensus.cc:993] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:08.385824 28124 raft_consensus.cc:1081] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:08.387295 27799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:08.387776 27799 raft_consensus.cc:493] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.388069 27799 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.393079 27799 raft_consensus.cc:515] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.394572 27799 leader_election.cc:290] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:08.395316 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:08.395659 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41a21f221f8d4a59b291f754f86ed190" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:08.395984 28062 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.396347 27875 raft_consensus.cc:3055] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.396658 27875 raft_consensus.cc:740] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:08.397639 27875 consensus_queue.cc:260] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.398680 27875 raft_consensus.cc:3060] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.401641 28062 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:08.402654 27767 leader_election.cc:304] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:08.403448 28148 raft_consensus.cc:2804] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.403941 28148 raft_consensus.cc:697] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:08.405032 27875 raft_consensus.cc:2468] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:08.404865 28148 consensus_queue.cc:237] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:08.412006 27624 catalog_manager.cc:5697] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.687548 27875 raft_consensus.cc:1275] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.688943 28160 consensus_queue.cc:1048] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.698194 28062 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.699973 28135 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.703967 28071 raft_consensus.cc:1275] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.706295 28160 consensus_queue.cc:1048] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.718820 27800 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.720430 28124 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:08.871408 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:08.964354 27875 raft_consensus.cc:1275] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.965844 28160 consensus_queue.cc:1048] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.976327 28071 raft_consensus.cc:1275] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.978433 28160 consensus_queue.cc:1048] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.040679 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:09.049428 27875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:09.050032 27875 raft_consensus.cc:606] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:09.052812 27800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:09.053442 27800 raft_consensus.cc:606] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:09.055938 27800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:09.056458 27800 raft_consensus.cc:606] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:09.057510 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:09.057982 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:09.058563 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:09.204320 28135 raft_consensus.cc:993] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:09.204919 28135 raft_consensus.cc:1081] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:09.206580 27800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:09.207186 27800 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:09.207571 27800 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.213219 27800 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.215305 27800 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 3 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:09.216099 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:09.216784 28062 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.216703 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:09.217581 27875 raft_consensus.cc:3055] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:09.218351 27875 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:09.219218 27875 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.220567 27875 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.224006 28062 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 3.
I20260709 12:21:09.225178 27767 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:09.225848 28160 raft_consensus.cc:2804] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:09.226317 28160 raft_consensus.cc:697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:09.226747 27875 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 3.
I20260709 12:21:09.227219 28160 consensus_queue.cc:237] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.234016 27624 catalog_manager.cc:5697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 2 to 3, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 3 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.499861 28116 raft_consensus.cc:993] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:09.500397 28160 raft_consensus.cc:1081] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:09.501883 27874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:09.502400 27874 raft_consensus.cc:493] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:09.502736 27874 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.507754 27874 raft_consensus.cc:515] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:09.509672 27874 leader_election.cc:290] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Requested vote from peers 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455), fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:21:09.510494 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:09.510802 28062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:09.511158 27800 raft_consensus.cc:3055] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:09.511430 28062 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.511538 27800 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:09.512465 27800 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:09.513701 27800 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.518539 28062 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:09.519726 27841 leader_election.cc:304] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:09.520428 28135 raft_consensus.cc:2804] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:09.520612 27800 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 2.
I20260709 12:21:09.520968 28135 raft_consensus.cc:697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:09.522075 28135 consensus_queue.cc:237] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:09.529196 27624 catalog_manager.cc:5697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 1 to 2, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:09.599586 28116 raft_consensus.cc:993] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:09.600121 28160 raft_consensus.cc:1081] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:09.601634 27875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:09.602130 27875 raft_consensus.cc:493] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:09.602461 27875 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.608590 27875 raft_consensus.cc:515] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.610195 27875 leader_election.cc:290] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 3 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:09.610903 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:09.611279 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef8f6210fb2541a4953756727e78be7d" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:09.611583 28071 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.611930 27800 raft_consensus.cc:3055] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:09.612270 27800 raft_consensus.cc:740] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:09.613045 27800 consensus_queue.cc:260] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.614097 27800 raft_consensus.cc:3060] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:09.618022 28071 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 3.
I20260709 12:21:09.619071 27841 leader_election.cc:304] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:09.619475 27800 raft_consensus.cc:2468] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 3.
I20260709 12:21:09.619836 28135 raft_consensus.cc:2804] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:09.620380 28135 raft_consensus.cc:697] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:09.621107 28135 consensus_queue.cc:237] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:09.627348 27624 catalog_manager.cc:5697] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 2 to 3, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:09.689976 28071 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:09.691314 28116 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:09.701385 27875 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:09.704777 28148 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:21:09.872460 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:09.987604 28071 raft_consensus.cc:1275] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.989009 28135 consensus_queue.cc:1048] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.999063 27800 raft_consensus.cc:1275] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:10.000912 28182 consensus_queue.cc:1048] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:10.065088 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.073318 27800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
mode: GRACEFUL
new_leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:34264
I20260709 12:21:10.074045 27800 raft_consensus.cc:606] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Received request to transfer leadership to TS 9382d4e917c248b49f2fc98a184bb5b2
I20260709 12:21:10.076962 27875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:10.077512 27875 raft_consensus.cc:606] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:10.079826 27875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
mode: GRACEFUL
new_leader_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:55300
I20260709 12:21:10.080431 27875 raft_consensus.cc:606] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Received request to transfer leadership to TS 6c16865bd6df4722b0c94e798d94448c
I20260709 12:21:10.081511 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:10.082075 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:10.082763 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.106806 27800 raft_consensus.cc:1275] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:10.108254 28182 consensus_queue.cc:1048] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:10.117470 28071 raft_consensus.cc:1275] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:10.119180 28124 consensus_queue.cc:1048] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:10.202795 28135 raft_consensus.cc:993] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:10.203444 28124 raft_consensus.cc:1081] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:10.205092 27800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:10.205546 27800 raft_consensus.cc:493] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:10.205793 27800 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:10.210853 27800 raft_consensus.cc:515] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.212484 27800 leader_election.cc:290] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009)
I20260709 12:21:10.213466 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:10.214075 27875 raft_consensus.cc:3055] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:10.214002 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc54a8122bd449c2b61edc79daaac43b" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:10.214566 27875 raft_consensus.cc:740] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:10.214769 28071 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:10.215643 27875 consensus_queue.cc:260] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.216868 27875 raft_consensus.cc:3060] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:10.219928 28071 raft_consensus.cc:2468] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:10.220886 27767 leader_election.cc:304] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:10.221544 28160 raft_consensus.cc:2804] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:10.222035 28160 raft_consensus.cc:697] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:10.222153 27875 raft_consensus.cc:2468] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 2.
I20260709 12:21:10.223003 28160 consensus_queue.cc:237] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.230100 27622 catalog_manager.cc:5697] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 1 to 2, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 2 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.279744 28148 raft_consensus.cc:993] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: : Instructing follower 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:10.280252 28191 raft_consensus.cc:1081] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Signalling peer 9382d4e917c248b49f2fc98a184bb5b2 to start an election
I20260709 12:21:10.281795 27875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e"
dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
 from {username='slave'} at 127.0.0.1:59086
I20260709 12:21:10.282296 27875 raft_consensus.cc:493] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:10.282598 27875 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:10.287822 27875 raft_consensus.cc:515] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.289340 27875 leader_election.cc:290] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 4 election: Requested vote from peers fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755), 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455)
I20260709 12:21:10.290115 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:10.290711 28071 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:10.290565 27800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea5e6d2c3f84a3bb8f8c3dcab6aa89e" candidate_uuid: "9382d4e917c248b49f2fc98a184bb5b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
I20260709 12:21:10.291265 27800 raft_consensus.cc:3055] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:10.291616 27800 raft_consensus.cc:740] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:10.292409 27800 consensus_queue.cc:260] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.293649 27800 raft_consensus.cc:3060] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:10.297725 28071 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 4.
I20260709 12:21:10.298992 27841 leader_election.cc:304] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382d4e917c248b49f2fc98a184bb5b2, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:10.299762 28124 raft_consensus.cc:2804] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:10.300175 28124 raft_consensus.cc:697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 4 LEADER]: Becoming Leader. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:10.300263 27800 raft_consensus.cc:2468] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382d4e917c248b49f2fc98a184bb5b2 in term 4.
I20260709 12:21:10.300995 28124 consensus_queue.cc:237] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } }
I20260709 12:21:10.307106 27622 catalog_manager.cc:5697] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 reported cstate change: term changed from 3 to 4, leader changed from 6c16865bd6df4722b0c94e798d94448c (127.16.193.66) to 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67). New cstate: current_term: 4 leader_uuid: "9382d4e917c248b49f2fc98a184bb5b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:10.471453 28182 raft_consensus.cc:993] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: : Instructing follower 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:10.471999 28124 raft_consensus.cc:1081] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Signalling peer 6c16865bd6df4722b0c94e798d94448c to start an election
I20260709 12:21:10.473754 27800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7"
dest_uuid: "6c16865bd6df4722b0c94e798d94448c"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:21:10.474406 27800 raft_consensus.cc:493] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:10.474831 27800 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:10.483301 27800 raft_consensus.cc:515] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:10.485499 27800 leader_election.cc:290] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 3 election: Requested vote from peers 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67:37009), fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755)
I20260709 12:21:10.486114 27875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9382d4e917c248b49f2fc98a184bb5b2"
I20260709 12:21:10.486863 27875 raft_consensus.cc:3055] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:10.487290 27875 raft_consensus.cc:740] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9382d4e917c248b49f2fc98a184bb5b2, State: Running, Role: LEADER
I20260709 12:21:10.488145 27875 consensus_queue.cc:260] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:10.489180 28071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e42dc433a5c54a96acb3f9db9f9ee2d7" candidate_uuid: "6c16865bd6df4722b0c94e798d94448c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb6b631b99cc4cd299b619ae5936c95e"
I20260709 12:21:10.490336 27875 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:10.490494 28071 raft_consensus.cc:3060] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:10.497458 28071 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 3.
I20260709 12:21:10.497467 27875 raft_consensus.cc:2468] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c16865bd6df4722b0c94e798d94448c in term 3.
I20260709 12:21:10.498754 27767 leader_election.cc:304] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c16865bd6df4722b0c94e798d94448c, fb6b631b99cc4cd299b619ae5936c95e; no voters: 
I20260709 12:21:10.499596 28191 raft_consensus.cc:2804] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:10.500082 28191 raft_consensus.cc:697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Becoming Leader. State: Replica: 6c16865bd6df4722b0c94e798d94448c, State: Running, Role: LEADER
I20260709 12:21:10.500943 28191 consensus_queue.cc:237] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } }
I20260709 12:21:10.508867 27622 catalog_manager.cc:5697] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c reported cstate change: term changed from 2 to 3, leader changed from 9382d4e917c248b49f2fc98a184bb5b2 (127.16.193.67) to 6c16865bd6df4722b0c94e798d94448c (127.16.193.66). New cstate: current_term: 3 leader_uuid: "6c16865bd6df4722b0c94e798d94448c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.647408 28071 raft_consensus.cc:1275] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:10.648620 28191 consensus_queue.cc:1048] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:10.658211 27875 raft_consensus.cc:1275] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:10.660144 28191 consensus_queue.cc:1048] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:10.779018 27800 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 4 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:10.780426 28124 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c16865bd6df4722b0c94e798d94448c" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:10.788861 28071 raft_consensus.cc:1275] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 4 FOLLOWER]: Refusing update from remote peer 9382d4e917c248b49f2fc98a184bb5b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:10.790350 28182 consensus_queue.cc:1048] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:21:10.873400 27672 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:10.928345 28071 raft_consensus.cc:1275] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:10.929912 28191 consensus_queue.cc:1048] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb6b631b99cc4cd299b619ae5936c95e" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:10.938104 27875 raft_consensus.cc:1275] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Refusing update from remote peer 6c16865bd6df4722b0c94e798d94448c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:10.939821 28148 consensus_queue.cc:1048] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382d4e917c248b49f2fc98a184bb5b2" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:11.090603 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:11.120199 17157 tablet_server.cc:179] TabletServer@127.16.193.65:41755 shutting down...
I20260709 12:21:11.139902 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:11.140661 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.141364 17157 raft_consensus.cc:2243] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.141902 17157 raft_consensus.cc:2272] T e42dc433a5c54a96acb3f9db9f9ee2d7 P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.144286 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.144874 17157 raft_consensus.cc:2243] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.145295 17157 raft_consensus.cc:2272] T cc54a8122bd449c2b61edc79daaac43b P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.147475 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.148013 17157 raft_consensus.cc:2243] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.148458 17157 raft_consensus.cc:2272] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P fb6b631b99cc4cd299b619ae5936c95e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.150796 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.151423 17157 raft_consensus.cc:2243] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.151856 17157 raft_consensus.cc:2272] T ef8f6210fb2541a4953756727e78be7d P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.153689 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.154358 17157 raft_consensus.cc:2243] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.154841 17157 raft_consensus.cc:2272] T 7926a3d51fe0495dae7b06a1e7b5e62a P fb6b631b99cc4cd299b619ae5936c95e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.156642 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.157155 17157 raft_consensus.cc:2243] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.157569 17157 raft_consensus.cc:2272] T 563e3f63dffc4ef9a95af236cdb46331 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.159351 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.159835 17157 raft_consensus.cc:2243] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.160230 17157 raft_consensus.cc:2272] T 069d81124dac48beba9a7ad4aae6a254 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.162027 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.162524 17157 raft_consensus.cc:2243] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.162935 17157 raft_consensus.cc:2272] T e9f826b0243048c78a2fb584d36c21a2 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.164767 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.165316 17157 raft_consensus.cc:2243] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.165717 17157 raft_consensus.cc:2272] T c16c7cfa0a84454baee93fbe49b9d922 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.167711 17157 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:11.168156 17157 raft_consensus.cc:2243] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.168555 17157 raft_consensus.cc:2272] T 41a21f221f8d4a59b291f754f86ed190 P fb6b631b99cc4cd299b619ae5936c95e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.187448 17157 tablet_server.cc:196] TabletServer@127.16.193.65:41755 shutdown complete.
W20260709 12:21:11.197443 27767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111) [suppressed 73 similar messages]
W20260709 12:21:11.200951 27767 consensus_peers.cc:597] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.206156 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:21:11.229162 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:11.229846 17157 tablet_replica.cc:333] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.230859 17157 raft_consensus.cc:2243] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.231874 17157 raft_consensus.cc:2272] T cc54a8122bd449c2b61edc79daaac43b P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.235626 17157 tablet_replica.cc:333] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.236382 17157 raft_consensus.cc:2243] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:21:11.236953 27838 consensus_peers.cc:597] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Couldn't send request to peer 6c16865bd6df4722b0c94e798d94448c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.237475 17157 raft_consensus.cc:2272] T c16c7cfa0a84454baee93fbe49b9d922 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.240875 17157 tablet_replica.cc:333] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.241640 17157 raft_consensus.cc:2243] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:21:11.242326 27841 consensus_peers.cc:597] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.242609 17157 raft_consensus.cc:2272] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.245375 17157 tablet_replica.cc:333] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.245898 17157 raft_consensus.cc:2243] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.246264 17157 raft_consensus.cc:2272] T ef8f6210fb2541a4953756727e78be7d P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.248103 17157 tablet_replica.cc:333] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.248562 17157 raft_consensus.cc:2243] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.248905 17157 raft_consensus.cc:2272] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 6c16865bd6df4722b0c94e798d94448c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.251531 17157 tablet_replica.cc:333] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.252218 17157 raft_consensus.cc:2243] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.253185 17157 raft_consensus.cc:2272] T e9f826b0243048c78a2fb584d36c21a2 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.255291 27838 consensus_peers.cc:597] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer 6c16865bd6df4722b0c94e798d94448c (127.16.193.66:36455): Couldn't send request to peer 6c16865bd6df4722b0c94e798d94448c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.256228 17157 tablet_replica.cc:333] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
W20260709 12:21:11.256270 27841 consensus_peers.cc:597] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.256969 17157 raft_consensus.cc:2243] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.257748 17157 raft_consensus.cc:2272] T 069d81124dac48beba9a7ad4aae6a254 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.259845 17157 tablet_replica.cc:333] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.260324 17157 raft_consensus.cc:2243] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.261092 17157 raft_consensus.cc:2272] T 41a21f221f8d4a59b291f754f86ed190 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.263064 17157 tablet_replica.cc:333] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.263664 17157 raft_consensus.cc:2243] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.264070 17157 raft_consensus.cc:2272] T 563e3f63dffc4ef9a95af236cdb46331 P 6c16865bd6df4722b0c94e798d94448c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.265985 17157 tablet_replica.cc:333] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c: stopping tablet replica
I20260709 12:21:11.266405 17157 raft_consensus.cc:2243] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.266816 17157 raft_consensus.cc:2272] T 7926a3d51fe0495dae7b06a1e7b5e62a P 6c16865bd6df4722b0c94e798d94448c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.296845 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:21:11.311673 27841 consensus_peers.cc:597] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 -> Peer fb6b631b99cc4cd299b619ae5936c95e (127.16.193.65:41755): Couldn't send request to peer fb6b631b99cc4cd299b619ae5936c95e. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:41755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.324429 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:21:11.352399 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:11.353188 17157 tablet_replica.cc:333] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.353981 17157 raft_consensus.cc:2243] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.354523 17157 raft_consensus.cc:2272] T c16c7cfa0a84454baee93fbe49b9d922 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.356873 17157 tablet_replica.cc:333] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.357573 17157 raft_consensus.cc:2243] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.358371 17157 raft_consensus.cc:2272] T 563e3f63dffc4ef9a95af236cdb46331 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.360824 17157 tablet_replica.cc:333] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.361455 17157 raft_consensus.cc:2243] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.362216 17157 raft_consensus.cc:2272] T 7926a3d51fe0495dae7b06a1e7b5e62a P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.364498 17157 tablet_replica.cc:333] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.365130 17157 raft_consensus.cc:2243] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.365561 17157 raft_consensus.cc:2272] T e9f826b0243048c78a2fb584d36c21a2 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.367796 17157 tablet_replica.cc:333] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.368353 17157 raft_consensus.cc:2243] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.368780 17157 raft_consensus.cc:2272] T 069d81124dac48beba9a7ad4aae6a254 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.370971 17157 tablet_replica.cc:333] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.371575 17157 raft_consensus.cc:2243] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.372396 17157 raft_consensus.cc:2272] T bea5e6d2c3f84a3bb8f8c3dcab6aa89e P 9382d4e917c248b49f2fc98a184bb5b2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.375676 17157 tablet_replica.cc:333] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.377231 17157 raft_consensus.cc:2243] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.378753 17157 raft_consensus.cc:2272] T ef8f6210fb2541a4953756727e78be7d P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.384385 17157 tablet_replica.cc:333] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.385746 17157 raft_consensus.cc:2243] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.386161 17157 raft_consensus.cc:2272] T 41a21f221f8d4a59b291f754f86ed190 P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.388340 17157 tablet_replica.cc:333] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.388819 17157 raft_consensus.cc:2243] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.389194 17157 raft_consensus.cc:2272] T cc54a8122bd449c2b61edc79daaac43b P 9382d4e917c248b49f2fc98a184bb5b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.392513 17157 tablet_replica.cc:333] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2: stopping tablet replica
I20260709 12:21:11.392988 17157 raft_consensus.cc:2243] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.393364 17157 raft_consensus.cc:2272] T e42dc433a5c54a96acb3f9db9f9ee2d7 P 9382d4e917c248b49f2fc98a184bb5b2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.426646 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:21:11.452241 17157 master.cc:562] Master@127.16.193.126:35859 shutting down...
I20260709 12:21:11.477409 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.477978 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.478336 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2e963625c6804f4d90b4e712d636fee2: stopping tablet replica
I20260709 12:21:11.499675 17157 master.cc:584] Master@127.16.193.126:35859 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12846 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:21:11.542753 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:43943
I20260709 12:21:11.543949 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:11.548939 28206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.549005 28207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.551532 28209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.552242 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:11.553128 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:11.553351 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:11.553506 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599671553489 us; error 0 us; skew 500 ppm
I20260709 12:21:11.554067 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:11.556356 17157 webserver.cc:533] Webserver started at http://127.16.193.126:43797/ using document root <none> and password file <none>
I20260709 12:21:11.556872 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:11.557083 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:11.557377 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:11.558506 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "169109097d76483996bac44c53745c99"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.562918 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:21:11.566236 28214 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.567109 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:11.567407 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "169109097d76483996bac44c53745c99"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.567665 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:11.587796 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:11.588922 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:11.627678 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:43943
I20260709 12:21:11.627807 28265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:43943 every 8 connection(s)
I20260709 12:21:11.631760 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:11.643072 28266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99: Bootstrap starting.
I20260709 12:21:11.647967 28266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:11.652254 28266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99: No bootstrap required, opened a new log
I20260709 12:21:11.654385 28266 raft_consensus.cc:359] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER }
I20260709 12:21:11.654778 28266 raft_consensus.cc:385] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:11.655009 28266 raft_consensus.cc:740] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169109097d76483996bac44c53745c99, State: Initialized, Role: FOLLOWER
I20260709 12:21:11.655702 28266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER }
I20260709 12:21:11.656167 28266 raft_consensus.cc:399] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:11.656390 28266 raft_consensus.cc:493] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:11.656649 28266 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:11.661253 28266 raft_consensus.cc:515] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER }
I20260709 12:21:11.661742 28266 leader_election.cc:304] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 169109097d76483996bac44c53745c99; no voters: 
I20260709 12:21:11.662907 28266 leader_election.cc:290] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:11.663369 28269 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:11.664858 28269 raft_consensus.cc:697] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 1 LEADER]: Becoming Leader. State: Replica: 169109097d76483996bac44c53745c99, State: Running, Role: LEADER
I20260709 12:21:11.665501 28269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER }
I20260709 12:21:11.665750 28266 sys_catalog.cc:565] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:11.670588 28270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "169109097d76483996bac44c53745c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER } }
I20260709 12:21:11.670991 28271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169109097d76483996bac44c53745c99. Latest consensus state: current_term: 1 leader_uuid: "169109097d76483996bac44c53745c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169109097d76483996bac44c53745c99" member_type: VOTER } }
I20260709 12:21:11.671523 28270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:11.671826 28271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:11.676247 28276 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:11.682124 28276 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:11.685375 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:11.691195 28276 catalog_manager.cc:1383] Generated new cluster ID: debdac6c5388483299ba7e79245126a9
I20260709 12:21:11.691526 28276 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:11.711628 28276 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:11.713006 28276 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:11.726905 28276 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99: Generated new TSK 0
I20260709 12:21:11.727581 28276 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:11.752489 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:11.759048 28287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.760052 28288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.761616 28290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.762362 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:11.763211 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:11.763454 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:11.763586 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599671763576 us; error 0 us; skew 500 ppm
I20260709 12:21:11.764070 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:11.766327 17157 webserver.cc:533] Webserver started at http://127.16.193.65:42553/ using document root <none> and password file <none>
I20260709 12:21:11.766784 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:11.767030 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:11.767302 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:11.768455 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "9ef89c5222014b6a9685d41bf172dc57"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.772948 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:21:11.776319 28295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.777181 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:21:11.777521 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "9ef89c5222014b6a9685d41bf172dc57"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.777812 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:11.801710 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:11.802946 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:11.804502 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:11.806865 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:11.807053 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.807387 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:11.807530 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.849840 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:39713
I20260709 12:21:11.849946 28357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:39713 every 8 connection(s)
I20260709 12:21:11.854826 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:11.861732 28362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.863521 28363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.865696 17157 server_base.cc:1061] running on GCE node
W20260709 12:21:11.866814 28365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.867893 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:21:11.868108 28358 heartbeater.cc:344] Connected to a master server at 127.16.193.126:43943
W20260709 12:21:11.868124 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:11.868561 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599671868541 us; error 0 us; skew 500 ppm
I20260709 12:21:11.868605 28358 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:11.869323 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:11.869558 28358 heartbeater.cc:507] Master 127.16.193.126:43943 requested a full tablet report, sending...
I20260709 12:21:11.872042 28231 ts_manager.cc:194] Registered new tserver with Master: 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65:39713)
I20260709 12:21:11.872056 17157 webserver.cc:533] Webserver started at http://127.16.193.66:44843/ using document root <none> and password file <none>
I20260709 12:21:11.872699 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:11.872943 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:11.873200 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:11.874043 28231 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55546
I20260709 12:21:11.874236 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.878980 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:21:11.882442 28370 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.883188 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:11.883551 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.883858 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:11.892613 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:11.893734 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:11.895151 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:11.897392 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:11.897614 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.897857 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:11.898027 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.940387 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:38655
I20260709 12:21:11.940508 28432 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:38655 every 8 connection(s)
I20260709 12:21:11.945320 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:11.952397 28436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:11.954280 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.957540 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:11.958066 28433 heartbeater.cc:344] Connected to a master server at 127.16.193.126:43943
I20260709 12:21:11.958573 28433 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:11.958973 28439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:11.959429 28433 heartbeater.cc:507] Master 127.16.193.126:43943 requested a full tablet report, sending...
I20260709 12:21:11.960121 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:11.960397 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:11.960589 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599671960572 us; error 0 us; skew 500 ppm
I20260709 12:21:11.961324 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:11.961681 28231 ts_manager.cc:194] Registered new tserver with Master: 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655)
I20260709 12:21:11.963567 28231 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55556
I20260709 12:21:11.963961 17157 webserver.cc:533] Webserver started at http://127.16.193.67:44589/ using document root <none> and password file <none>
I20260709 12:21:11.964658 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:11.964941 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:11.965215 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:11.966405 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "735dea569f5946d486e280f24cb73614"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.971158 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:21:11.974658 28444 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:11.975757 17157 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:11.976069 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "735dea569f5946d486e280f24cb73614"
format_stamp: "Formatted at 2026-07-09 12:21:11 on dist-test-slave-n640"
I20260709 12:21:11.976428 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:12.003070 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:12.004565 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:12.006289 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:12.008661 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:12.008883 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:12.009119 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:12.009334 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:12.048889 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:37271
I20260709 12:21:12.049002 28506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:37271 every 8 connection(s)
I20260709 12:21:12.063061 28507 heartbeater.cc:344] Connected to a master server at 127.16.193.126:43943
I20260709 12:21:12.063597 28507 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:12.064311 28507 heartbeater.cc:507] Master 127.16.193.126:43943 requested a full tablet report, sending...
I20260709 12:21:12.066489 28231 ts_manager.cc:194] Registered new tserver with Master: 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.067126 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014742088s
I20260709 12:21:12.068127 28231 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55570
I20260709 12:21:12.093482 28231 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55572:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:12.095759 28231 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:12.140816 28472 tablet_service.cc:1511] Processing CreateTablet for tablet ddb7dcb8d7a64c61bda27fd05252fe6a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.140965 28471 tablet_service.cc:1511] Processing CreateTablet for tablet 114245b96e1d4e0588eabaaba3adcb48 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.142488 28472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb7dcb8d7a64c61bda27fd05252fe6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.143249 28471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114245b96e1d4e0588eabaaba3adcb48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.152254 28398 tablet_service.cc:1511] Processing CreateTablet for tablet ddb7dcb8d7a64c61bda27fd05252fe6a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.153702 28398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb7dcb8d7a64c61bda27fd05252fe6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.158702 28397 tablet_service.cc:1511] Processing CreateTablet for tablet 114245b96e1d4e0588eabaaba3adcb48 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.160112 28397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114245b96e1d4e0588eabaaba3adcb48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.171159 28323 tablet_service.cc:1511] Processing CreateTablet for tablet ddb7dcb8d7a64c61bda27fd05252fe6a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.172739 28323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddb7dcb8d7a64c61bda27fd05252fe6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.176606 28322 tablet_service.cc:1511] Processing CreateTablet for tablet 114245b96e1d4e0588eabaaba3adcb48 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=ed98fa563ba044669c3823c9e023d084]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.178287 28322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114245b96e1d4e0588eabaaba3adcb48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.200799 28521 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: Bootstrap starting.
I20260709 12:21:12.200799 28522 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: Bootstrap starting.
I20260709 12:21:12.203382 28523 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: Bootstrap starting.
I20260709 12:21:12.209260 28522 tablet_bootstrap.cc:654] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.210157 28521 tablet_bootstrap.cc:654] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.210866 28523 tablet_bootstrap.cc:654] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.214485 28522 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: No bootstrap required, opened a new log
I20260709 12:21:12.215051 28522 ts_tablet_manager.cc:1403] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 12:21:12.215960 28521 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: No bootstrap required, opened a new log
I20260709 12:21:12.216391 28521 ts_tablet_manager.cc:1403] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:21:12.217489 28522 raft_consensus.cc:359] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.218152 28522 raft_consensus.cc:385] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.218469 28522 raft_consensus.cc:740] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.218660 28521 raft_consensus.cc:359] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.219511 28521 raft_consensus.cc:385] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.219357 28522 consensus_queue.cc:260] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.219914 28521 raft_consensus.cc:740] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735dea569f5946d486e280f24cb73614, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.220391 28523 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: No bootstrap required, opened a new log
I20260709 12:21:12.220705 28521 consensus_queue.cc:260] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.221910 28433 heartbeater.cc:499] Master 127.16.193.126:43943 was elected leader, sending a full tablet report...
I20260709 12:21:12.222491 28522 ts_tablet_manager.cc:1434] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:12.220935 28523 ts_tablet_manager.cc:1403] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.008s
I20260709 12:21:12.224555 28507 heartbeater.cc:499] Master 127.16.193.126:43943 was elected leader, sending a full tablet report...
I20260709 12:21:12.224764 28522 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: Bootstrap starting.
I20260709 12:21:12.225849 28523 raft_consensus.cc:359] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.227741 28521 ts_tablet_manager.cc:1434] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:12.228133 28523 raft_consensus.cc:385] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.228535 28523 raft_consensus.cc:740] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.229071 28521 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: Bootstrap starting.
I20260709 12:21:12.229530 28523 consensus_queue.cc:260] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.234220 28522 tablet_bootstrap.cc:654] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.237684 28521 tablet_bootstrap.cc:654] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.253787 28521 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: No bootstrap required, opened a new log
I20260709 12:21:12.254017 28522 tablet_bootstrap.cc:492] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: No bootstrap required, opened a new log
I20260709 12:21:12.254467 28521 ts_tablet_manager.cc:1403] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.014s
I20260709 12:21:12.254478 28358 heartbeater.cc:499] Master 127.16.193.126:43943 was elected leader, sending a full tablet report...
I20260709 12:21:12.254693 28522 ts_tablet_manager.cc:1403] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.000s
I20260709 12:21:12.255079 28523 ts_tablet_manager.cc:1434] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: Time spent starting tablet: real 0.032s	user 0.016s	sys 0.010s
I20260709 12:21:12.256238 28523 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: Bootstrap starting.
I20260709 12:21:12.257827 28521 raft_consensus.cc:359] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.258150 28522 raft_consensus.cc:359] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.258658 28521 raft_consensus.cc:385] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.258981 28522 raft_consensus.cc:385] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.259119 28521 raft_consensus.cc:740] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735dea569f5946d486e280f24cb73614, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.259516 28522 raft_consensus.cc:740] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.260267 28521 consensus_queue.cc:260] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.260465 28522 consensus_queue.cc:260] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.262630 28521 ts_tablet_manager.cc:1434] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:12.263329 28522 ts_tablet_manager.cc:1434] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:12.266438 28523 tablet_bootstrap.cc:654] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.276482 28523 tablet_bootstrap.cc:492] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: No bootstrap required, opened a new log
I20260709 12:21:12.276916 28523 ts_tablet_manager.cc:1403] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.002s
I20260709 12:21:12.279515 28523 raft_consensus.cc:359] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.280068 28523 raft_consensus.cc:385] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.280316 28523 raft_consensus.cc:740] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.280987 28523 consensus_queue.cc:260] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.282938 28523 ts_tablet_manager.cc:1434] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:12.308118 28529 raft_consensus.cc:493] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:12.308535 28529 raft_consensus.cc:515] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.310767 28529 leader_election.cc:290] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271), 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655)
I20260709 12:21:12.320149 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb7dcb8d7a64c61bda27fd05252fe6a" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614" is_pre_election: true
I20260709 12:21:12.320870 28482 raft_consensus.cc:2468] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.321388 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb7dcb8d7a64c61bda27fd05252fe6a" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" is_pre_election: true
I20260709 12:21:12.322125 28296 leader_election.cc:304] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735dea569f5946d486e280f24cb73614, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.322127 28408 raft_consensus.cc:2468] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.322976 28529 raft_consensus.cc:2804] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:12.323477 28529 raft_consensus.cc:493] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:12.323745 28529 raft_consensus.cc:3060] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.328629 28529 raft_consensus.cc:515] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.330274 28529 leader_election.cc:290] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Requested vote from peers 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271), 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655)
I20260709 12:21:12.330963 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb7dcb8d7a64c61bda27fd05252fe6a" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:12.331324 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddb7dcb8d7a64c61bda27fd05252fe6a" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
I20260709 12:21:12.331620 28482 raft_consensus.cc:3060] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.331930 28408 raft_consensus.cc:3060] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.339262 28482 raft_consensus.cc:2468] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.339262 28408 raft_consensus.cc:2468] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.340673 28296 leader_election.cc:304] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735dea569f5946d486e280f24cb73614, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.341470 28529 raft_consensus.cc:2804] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:12.342677 28529 raft_consensus.cc:697] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Becoming Leader. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:12.343603 28529 consensus_queue.cc:237] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } }
I20260709 12:21:12.350003 28229 catalog_manager.cc:5697] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "9ef89c5222014b6a9685d41bf172dc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.430091 28529 raft_consensus.cc:493] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:12.430704 28529 raft_consensus.cc:515] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.432786 28529 leader_election.cc:290] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.433791 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" is_pre_election: true
I20260709 12:21:12.433910 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614" is_pre_election: true
I20260709 12:21:12.434691 28408 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.434756 28482 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.436102 28298 leader_election.cc:304] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.436826 28529 raft_consensus.cc:2804] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:12.437121 28529 raft_consensus.cc:493] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:12.437397 28529 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.442281 28529 raft_consensus.cc:515] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.443861 28529 leader_election.cc:290] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Requested vote from peers 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.444561 28408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
I20260709 12:21:12.444912 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:12.445235 28408 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.445487 28482 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.452333 28482 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.452333 28408 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.453449 28298 leader_election.cc:304] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.454080 28529 raft_consensus.cc:2804] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:12.454526 28529 raft_consensus.cc:697] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Becoming Leader. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:12.455154 28529 consensus_queue.cc:237] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.461572 28229 catalog_manager.cc:5697] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "9ef89c5222014b6a9685d41bf172dc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.545706 28229 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55588:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:12.547797 28229 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:12.571919 28322 tablet_service.cc:1511] Processing CreateTablet for tablet 564303703cc64706a7169014c06be0e5 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.573262 28322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 564303703cc64706a7169014c06be0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.573267 28398 tablet_service.cc:1511] Processing CreateTablet for tablet 564303703cc64706a7169014c06be0e5 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.574699 28398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 564303703cc64706a7169014c06be0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.576565 28323 tablet_service.cc:1511] Processing CreateTablet for tablet f1a95beae3b34aa0b8786f7610f562af (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.577790 28323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a95beae3b34aa0b8786f7610f562af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.578383 28472 tablet_service.cc:1511] Processing CreateTablet for tablet 564303703cc64706a7169014c06be0e5 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:12.580221 28397 tablet_service.cc:1511] Processing CreateTablet for tablet f1a95beae3b34aa0b8786f7610f562af (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.581482 28397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a95beae3b34aa0b8786f7610f562af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.582279 28472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 564303703cc64706a7169014c06be0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.583266 28471 tablet_service.cc:1511] Processing CreateTablet for tablet f1a95beae3b34aa0b8786f7610f562af (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ded4227b0d814f31b235e4413e48111c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:12.584481 28471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a95beae3b34aa0b8786f7610f562af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.589793 28523 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: Bootstrap starting.
I20260709 12:21:12.596159 28523 tablet_bootstrap.cc:654] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.597031 28522 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: Bootstrap starting.
I20260709 12:21:12.606344 28523 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: No bootstrap required, opened a new log
I20260709 12:21:12.606910 28523 ts_tablet_manager.cc:1403] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:21:12.610193 28523 raft_consensus.cc:359] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.610896 28523 raft_consensus.cc:385] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.611269 28523 raft_consensus.cc:740] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.612269 28523 consensus_queue.cc:260] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.617353 28523 ts_tablet_manager.cc:1434] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:21:12.619649 28523 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: Bootstrap starting.
I20260709 12:21:12.622824 28521 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: Bootstrap starting.
I20260709 12:21:12.624892 28523 tablet_bootstrap.cc:654] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.628224 28522 tablet_bootstrap.cc:654] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.638345 28521 tablet_bootstrap.cc:654] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.639034 28522 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: No bootstrap required, opened a new log
I20260709 12:21:12.639549 28522 ts_tablet_manager.cc:1403] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.002s
I20260709 12:21:12.640486 28523 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: No bootstrap required, opened a new log
I20260709 12:21:12.640976 28523 ts_tablet_manager.cc:1403] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:21:12.642089 28522 raft_consensus.cc:359] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.642800 28522 raft_consensus.cc:385] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.643071 28522 raft_consensus.cc:740] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.643896 28521 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: No bootstrap required, opened a new log
I20260709 12:21:12.643855 28522 consensus_queue.cc:260] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.643985 28523 raft_consensus.cc:359] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.644531 28521 ts_tablet_manager.cc:1403] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.006s
I20260709 12:21:12.644876 28523 raft_consensus.cc:385] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.645205 28523 raft_consensus.cc:740] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.645957 28523 consensus_queue.cc:260] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.645959 28522 ts_tablet_manager.cc:1434] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:12.647892 28522 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: Bootstrap starting.
I20260709 12:21:12.648295 28523 ts_tablet_manager.cc:1434] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:12.650604 28521 raft_consensus.cc:359] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.651405 28521 raft_consensus.cc:385] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.651695 28521 raft_consensus.cc:740] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735dea569f5946d486e280f24cb73614, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.652459 28521 consensus_queue.cc:260] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.654621 28522 tablet_bootstrap.cc:654] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.654736 28521 ts_tablet_manager.cc:1434] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:21:12.655822 28521 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Bootstrap starting.
I20260709 12:21:12.662382 28521 tablet_bootstrap.cc:654] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.670100 28522 tablet_bootstrap.cc:492] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: No bootstrap required, opened a new log
I20260709 12:21:12.670548 28522 ts_tablet_manager.cc:1403] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.010s
I20260709 12:21:12.670766 28521 tablet_bootstrap.cc:492] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: No bootstrap required, opened a new log
I20260709 12:21:12.671303 28521 ts_tablet_manager.cc:1403] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.007s
I20260709 12:21:12.672808 28522 raft_consensus.cc:359] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.673493 28522 raft_consensus.cc:385] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.673316 28521 raft_consensus.cc:359] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.673834 28522 raft_consensus.cc:740] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.674095 28521 raft_consensus.cc:385] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.674422 28521 raft_consensus.cc:740] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735dea569f5946d486e280f24cb73614, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.674557 28522 consensus_queue.cc:260] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.675032 28521 consensus_queue.cc:260] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.676759 28522 ts_tablet_manager.cc:1434] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:21:12.677223 28521 ts_tablet_manager.cc:1434] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
W20260709 12:21:12.679675 28279 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:12.816023 28527 raft_consensus.cc:493] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:12.816649 28527 raft_consensus.cc:515] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.818744 28527 leader_election.cc:290] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65:39713), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.839745 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614" is_pre_election: true
I20260709 12:21:12.840495 28482 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 0.
I20260709 12:21:12.841531 28371 leader_election.cc:304] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 735dea569f5946d486e280f24cb73614; no voters: 
I20260709 12:21:12.841601 28333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef89c5222014b6a9685d41bf172dc57" is_pre_election: true
I20260709 12:21:12.842360 28527 raft_consensus.cc:2804] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:12.842549 28333 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 0.
I20260709 12:21:12.842752 28527 raft_consensus.cc:493] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:12.841679 28529 consensus_queue.cc:1048] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:12.843183 28527 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.845001 28529 consensus_queue.cc:1048] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:12.849933 28527 raft_consensus.cc:515] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.852363 28527 leader_election.cc:290] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 1 election: Requested vote from peers 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65:39713), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.853547 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:12.854142 28482 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.855563 28333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
I20260709 12:21:12.856174 28333 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.861361 28482 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 1.
I20260709 12:21:12.862816 28371 leader_election.cc:304] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 735dea569f5946d486e280f24cb73614; no voters: 
I20260709 12:21:12.864271 28333 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 1.
I20260709 12:21:12.868291 28552 raft_consensus.cc:2804] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:12.872579 28552 raft_consensus.cc:697] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 LEADER]: Becoming Leader. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Running, Role: LEADER
I20260709 12:21:12.873513 28552 consensus_queue.cc:237] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.877863 28534 consensus_queue.cc:1048] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:12.886940 28560 consensus_queue.cc:1048] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:12.898504 28560 raft_consensus.cc:493] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:12.899084 28560 raft_consensus.cc:515] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.898710 28228 catalog_manager.cc:5697] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.902819 28560 leader_election.cc:290] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.903541 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "564303703cc64706a7169014c06be0e5" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" is_pre_election: true
I20260709 12:21:12.904309 28407 raft_consensus.cc:2468] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.905534 28298 leader_election.cc:304] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.906554 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "564303703cc64706a7169014c06be0e5" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614" is_pre_election: true
I20260709 12:21:12.906754 28560 raft_consensus.cc:2804] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:12.907272 28560 raft_consensus.cc:493] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:12.907294 28482 raft_consensus.cc:2468] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 0.
I20260709 12:21:12.907660 28560 raft_consensus.cc:3060] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.913781 28560 raft_consensus.cc:515] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.915575 28560 leader_election.cc:290] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Requested vote from peers 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:12.916422 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "564303703cc64706a7169014c06be0e5" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
I20260709 12:21:12.916630 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "564303703cc64706a7169014c06be0e5" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:12.917070 28407 raft_consensus.cc:3060] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.917181 28482 raft_consensus.cc:3060] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.924450 28482 raft_consensus.cc:2468] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.924451 28407 raft_consensus.cc:2468] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 1.
I20260709 12:21:12.925724 28298 leader_election.cc:304] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:12.926476 28560 raft_consensus.cc:2804] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:12.926915 28560 raft_consensus.cc:697] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Becoming Leader. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:12.927841 28560 consensus_queue.cc:237] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:12.934702 28231 catalog_manager.cc:5697] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "9ef89c5222014b6a9685d41bf172dc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.010885 28333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48"
dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
mode: GRACEFUL
new_leader_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
 from {username='slave'} at 127.0.0.1:42730
I20260709 12:21:13.011482 28333 raft_consensus.cc:606] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Received request to transfer leadership to TS 25dc0977cf8c4dc8b6db2f629daf72d7
I20260709 12:21:13.028020 28407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af"
dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
mode: GRACEFUL
new_leader_uuid: "9ef89c5222014b6a9685d41bf172dc57"
 from {username='slave'} at 127.0.0.1:40926
I20260709 12:21:13.028581 28407 raft_consensus.cc:606] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 LEADER]: Received request to transfer leadership to TS 9ef89c5222014b6a9685d41bf172dc57
I20260709 12:21:13.334157 28560 consensus_queue.cc:1048] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:13.348868 28558 consensus_queue.cc:1048] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:13.360860 28527 consensus_queue.cc:1048] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:13.375602 28527 consensus_queue.cc:1048] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:13.384249 28552 raft_consensus.cc:993] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: : Instructing follower 9ef89c5222014b6a9685d41bf172dc57 to start an election
I20260709 12:21:13.384709 28527 raft_consensus.cc:1081] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 LEADER]: Signalling peer 9ef89c5222014b6a9685d41bf172dc57 to start an election
I20260709 12:21:13.386701 28333 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af"
dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
 from {username='slave'} at 127.0.0.1:42720
I20260709 12:21:13.387390 28333 raft_consensus.cc:493] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.387920 28333 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.397272 28333 raft_consensus.cc:515] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.399819 28333 leader_election.cc:290] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 2 election: Requested vote from peers 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:13.400244 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
I20260709 12:21:13.400496 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "9ef89c5222014b6a9685d41bf172dc57" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:13.401036 28407 raft_consensus.cc:3055] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.401197 28482 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.401445 28407 raft_consensus.cc:740] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Running, Role: LEADER
I20260709 12:21:13.402558 28407 consensus_queue.cc:260] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.403743 28407 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.410888 28407 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 2.
I20260709 12:21:13.411044 28482 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef89c5222014b6a9685d41bf172dc57 in term 2.
I20260709 12:21:13.412288 28298 leader_election.cc:304] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:13.413029 28558 raft_consensus.cc:2804] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.413477 28558 raft_consensus.cc:697] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 LEADER]: Becoming Leader. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:13.414347 28558 consensus_queue.cc:237] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.420274 28229 catalog_manager.cc:5697] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 reported cstate change: term changed from 1 to 2, leader changed from 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66) to 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "9ef89c5222014b6a9685d41bf172dc57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.435880 28529 raft_consensus.cc:993] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: : Instructing follower 25dc0977cf8c4dc8b6db2f629daf72d7 to start an election
I20260709 12:21:13.436362 28558 raft_consensus.cc:1081] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Signalling peer 25dc0977cf8c4dc8b6db2f629daf72d7 to start an election
I20260709 12:21:13.438019 28407 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48"
dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
 from {username='slave'} at 127.0.0.1:40916
I20260709 12:21:13.438488 28407 raft_consensus.cc:493] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.438778 28407 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.443848 28407 raft_consensus.cc:515] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.445564 28407 leader_election.cc:290] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 2 election: Requested vote from peers 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65:39713), 735dea569f5946d486e280f24cb73614 (127.16.193.67:37271)
I20260709 12:21:13.446458 28333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
I20260709 12:21:13.446653 28482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114245b96e1d4e0588eabaaba3adcb48" candidate_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "735dea569f5946d486e280f24cb73614"
I20260709 12:21:13.447173 28333 raft_consensus.cc:3055] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.447286 28482 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.447520 28333 raft_consensus.cc:740] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:13.448622 28333 consensus_queue.cc:260] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.450099 28333 raft_consensus.cc:3060] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.455082 28482 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 2.
I20260709 12:21:13.456784 28371 leader_election.cc:304] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25dc0977cf8c4dc8b6db2f629daf72d7, 735dea569f5946d486e280f24cb73614; no voters: 
I20260709 12:21:13.457284 28333 raft_consensus.cc:2468] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25dc0977cf8c4dc8b6db2f629daf72d7 in term 2.
I20260709 12:21:13.457854 28527 raft_consensus.cc:2804] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.458429 28527 raft_consensus.cc:697] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 LEADER]: Becoming Leader. State: Replica: 25dc0977cf8c4dc8b6db2f629daf72d7, State: Running, Role: LEADER
I20260709 12:21:13.459506 28527 consensus_queue.cc:237] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.466003 28229 catalog_manager.cc:5697] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 reported cstate change: term changed from 1 to 2, leader changed from 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65) to 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.747824 17157 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:21:13.749473 17157 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:21:13.749909 17157 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:21:13.763808 28333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af"
dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
mode: GRACEFUL
new_leader_uuid: "735dea569f5946d486e280f24cb73614"
 from {username='slave'} at 127.0.0.1:54086
I20260709 12:21:13.764420 28333 raft_consensus.cc:606] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 LEADER]: Received request to transfer leadership to TS 735dea569f5946d486e280f24cb73614
I20260709 12:21:13.765602 17157 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:21:13.871578 28407 raft_consensus.cc:1275] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Refusing update from remote peer 9ef89c5222014b6a9685d41bf172dc57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.873085 28558 consensus_queue.cc:1048] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.880903 28482 raft_consensus.cc:1275] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Refusing update from remote peer 9ef89c5222014b6a9685d41bf172dc57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.882550 28558 consensus_queue.cc:1048] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.897378 28575 raft_consensus.cc:993] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: : Instructing follower 735dea569f5946d486e280f24cb73614 to start an election
I20260709 12:21:13.897903 28575 raft_consensus.cc:1081] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 LEADER]: Signalling peer 735dea569f5946d486e280f24cb73614 to start an election
I20260709 12:21:13.900283 28482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af"
dest_uuid: "735dea569f5946d486e280f24cb73614"
 from {username='slave'} at 127.0.0.1:53726
I20260709 12:21:13.900926 28482 raft_consensus.cc:493] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.901362 28482 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:13.902408 28481 raft_consensus.cc:1275] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Refusing update from remote peer 25dc0977cf8c4dc8b6db2f629daf72d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.903952 28527 consensus_queue.cc:1048] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.909894 28482 raft_consensus.cc:515] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.918453 28333 raft_consensus.cc:1275] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Refusing update from remote peer 25dc0977cf8c4dc8b6db2f629daf72d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.919936 28482 leader_election.cc:290] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [CANDIDATE]: Term 3 election: Requested vote from peers 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65:39713), 25dc0977cf8c4dc8b6db2f629daf72d7 (127.16.193.66:38655)
I20260709 12:21:13.920195 28552 consensus_queue.cc:1048] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.938892 28332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "735dea569f5946d486e280f24cb73614" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9ef89c5222014b6a9685d41bf172dc57"
I20260709 12:21:13.939711 28332 raft_consensus.cc:3055] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:13.940109 28332 raft_consensus.cc:740] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9ef89c5222014b6a9685d41bf172dc57, State: Running, Role: LEADER
I20260709 12:21:13.940960 28332 consensus_queue.cc:260] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.942394 28332 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:13.949553 28407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a95beae3b34aa0b8786f7610f562af" candidate_uuid: "735dea569f5946d486e280f24cb73614" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7"
I20260709 12:21:13.950158 28407 raft_consensus.cc:3060] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:13.951524 28332 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735dea569f5946d486e280f24cb73614 in term 3.
I20260709 12:21:13.953151 28447 leader_election.cc:304] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 735dea569f5946d486e280f24cb73614, 9ef89c5222014b6a9685d41bf172dc57; no voters: 
I20260709 12:21:13.954120 28577 raft_consensus.cc:2804] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:13.955627 28577 raft_consensus.cc:697] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 LEADER]: Becoming Leader. State: Replica: 735dea569f5946d486e280f24cb73614, State: Running, Role: LEADER
I20260709 12:21:13.956737 28577 consensus_queue.cc:237] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } }
I20260709 12:21:13.957757 28407 raft_consensus.cc:2468] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 735dea569f5946d486e280f24cb73614 in term 3.
I20260709 12:21:13.963567 28228 catalog_manager.cc:5697] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 reported cstate change: term changed from 2 to 3, leader changed from 9ef89c5222014b6a9685d41bf172dc57 (127.16.193.65) to 735dea569f5946d486e280f24cb73614 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "735dea569f5946d486e280f24cb73614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef89c5222014b6a9685d41bf172dc57" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25dc0977cf8c4dc8b6db2f629daf72d7" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "735dea569f5946d486e280f24cb73614" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 37271 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.988451 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
I20260709 12:21:14.017287 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:14.018108 17157 tablet_replica.cc:333] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57: stopping tablet replica
I20260709 12:21:14.018946 17157 raft_consensus.cc:2243] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.019500 17157 raft_consensus.cc:2272] T f1a95beae3b34aa0b8786f7610f562af P 9ef89c5222014b6a9685d41bf172dc57 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.022456 17157 tablet_replica.cc:333] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57: stopping tablet replica
I20260709 12:21:14.023125 17157 raft_consensus.cc:2243] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.024101 17157 raft_consensus.cc:2272] T 564303703cc64706a7169014c06be0e5 P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.027457 17157 tablet_replica.cc:333] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57: stopping tablet replica
I20260709 12:21:14.028023 17157 raft_consensus.cc:2243] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.028916 17157 raft_consensus.cc:2272] T ddb7dcb8d7a64c61bda27fd05252fe6a P 9ef89c5222014b6a9685d41bf172dc57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.031313 17157 tablet_replica.cc:333] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57: stopping tablet replica
I20260709 12:21:14.031805 17157 raft_consensus.cc:2243] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.032229 17157 raft_consensus.cc:2272] T 114245b96e1d4e0588eabaaba3adcb48 P 9ef89c5222014b6a9685d41bf172dc57 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.058286 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
I20260709 12:21:14.077461 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:21:14.106123 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:14.106901 17157 tablet_replica.cc:333] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7: stopping tablet replica
I20260709 12:21:14.107676 17157 raft_consensus.cc:2243] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.108168 17157 raft_consensus.cc:2272] T 564303703cc64706a7169014c06be0e5 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.110925 17157 tablet_replica.cc:333] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7: stopping tablet replica
I20260709 12:21:14.111577 17157 raft_consensus.cc:2243] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.112018 17157 raft_consensus.cc:2272] T f1a95beae3b34aa0b8786f7610f562af P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.114748 17157 tablet_replica.cc:333] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7: stopping tablet replica
I20260709 12:21:14.115377 17157 raft_consensus.cc:2243] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.116204 17157 raft_consensus.cc:2272] T 114245b96e1d4e0588eabaaba3adcb48 P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.118798 17157 tablet_replica.cc:333] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7: stopping tablet replica
I20260709 12:21:14.119421 17157 raft_consensus.cc:2243] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.119880 17157 raft_consensus.cc:2272] T ddb7dcb8d7a64c61bda27fd05252fe6a P 25dc0977cf8c4dc8b6db2f629daf72d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.144757 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
I20260709 12:21:14.161356 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:21:14.183696 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:14.184464 17157 tablet_replica.cc:333] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: stopping tablet replica
I20260709 12:21:14.185132 17157 raft_consensus.cc:2243] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.185774 17157 pending_rounds.cc:70] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Trying to abort 1 pending ops.
I20260709 12:21:14.185984 17157 pending_rounds.cc:77] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305624264536768512 op_type: NO_OP noop_request { }
I20260709 12:21:14.186285 17157 raft_consensus.cc:2889] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:21:14.186584 17157 raft_consensus.cc:2272] T f1a95beae3b34aa0b8786f7610f562af P 735dea569f5946d486e280f24cb73614 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.189479 17157 tablet_replica.cc:333] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614: stopping tablet replica
I20260709 12:21:14.190066 17157 raft_consensus.cc:2243] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.190508 17157 raft_consensus.cc:2272] T 564303703cc64706a7169014c06be0e5 P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.193267 17157 tablet_replica.cc:333] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614: stopping tablet replica
I20260709 12:21:14.193760 17157 raft_consensus.cc:2243] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.194200 17157 raft_consensus.cc:2272] T 114245b96e1d4e0588eabaaba3adcb48 P 735dea569f5946d486e280f24cb73614 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.196735 17157 tablet_replica.cc:333] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614: stopping tablet replica
I20260709 12:21:14.197213 17157 raft_consensus.cc:2243] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.197647 17157 raft_consensus.cc:2272] T ddb7dcb8d7a64c61bda27fd05252fe6a P 735dea569f5946d486e280f24cb73614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.221814 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:21:14.240051 17157 master.cc:562] Master@127.16.193.126:43943 shutting down...
I20260709 12:21:14.260149 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.260707 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.261026 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 169109097d76483996bac44c53745c99: stopping tablet replica
I20260709 12:21:14.281477 17157 master.cc:584] Master@127.16.193.126:43943 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2777 ms)
[----------] 6 tests from LeaderRebalancerTest (361810 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:21:14.320483 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:36829
I20260709 12:21:14.321578 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:14.328150 28585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.328091 28586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.329322 28588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.330739 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:14.331691 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:14.331894 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:14.332073 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599674332040 us; error 0 us; skew 500 ppm
I20260709 12:21:14.332644 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:14.335182 17157 webserver.cc:533] Webserver started at http://127.16.193.126:43737/ using document root <none> and password file <none>
I20260709 12:21:14.335800 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:14.335973 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:14.336335 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:14.337503 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "b78b1e4c52dc481d9ba2740dc73b2277"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.342175 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:14.345594 28593 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.346376 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:21:14.346671 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "b78b1e4c52dc481d9ba2740dc73b2277"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.346917 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:14.361059 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:14.362300 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:14.400470 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:36829
I20260709 12:21:14.400578 28644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:36829 every 8 connection(s)
I20260709 12:21:14.404448 28645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.415751 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: Bootstrap starting.
I20260709 12:21:14.421087 28645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.425436 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: No bootstrap required, opened a new log
I20260709 12:21:14.427666 28645 raft_consensus.cc:359] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER }
I20260709 12:21:14.428028 28645 raft_consensus.cc:385] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.428253 28645 raft_consensus.cc:740] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b78b1e4c52dc481d9ba2740dc73b2277, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.428821 28645 consensus_queue.cc:260] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER }
I20260709 12:21:14.429406 28645 raft_consensus.cc:399] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:14.429646 28645 raft_consensus.cc:493] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:14.429893 28645 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.434589 28645 raft_consensus.cc:515] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER }
I20260709 12:21:14.435039 28645 leader_election.cc:304] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b78b1e4c52dc481d9ba2740dc73b2277; no voters: 
I20260709 12:21:14.436295 28645 leader_election.cc:290] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:14.436753 28648 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.438086 28648 raft_consensus.cc:697] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 1 LEADER]: Becoming Leader. State: Replica: b78b1e4c52dc481d9ba2740dc73b2277, State: Running, Role: LEADER
I20260709 12:21:14.438745 28648 consensus_queue.cc:237] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER }
I20260709 12:21:14.439311 28645 sys_catalog.cc:565] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:14.441761 28649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER } }
I20260709 12:21:14.441922 28650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b78b1e4c52dc481d9ba2740dc73b2277. Latest consensus state: current_term: 1 leader_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78b1e4c52dc481d9ba2740dc73b2277" member_type: VOTER } }
I20260709 12:21:14.442524 28650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:14.443032 28649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:14.445744 28654 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:14.453779 28654 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:14.455015 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:14.463088 28654 catalog_manager.cc:1383] Generated new cluster ID: 1e4de3a29b8f41ffba25975de9065e8d
I20260709 12:21:14.463403 28654 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:14.481694 28654 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:14.483039 28654 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:14.497025 28654 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: Generated new TSK 0
I20260709 12:21:14.497670 28654 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:14.521800 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:14.527869 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.529274 28667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.531689 17157 server_base.cc:1061] running on GCE node
W20260709 12:21:14.531939 28669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.533056 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:14.533294 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:14.533483 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599674533469 us; error 0 us; skew 500 ppm
I20260709 12:21:14.534075 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:14.536463 17157 webserver.cc:533] Webserver started at http://127.16.193.65:41403/ using document root <none> and password file <none>
I20260709 12:21:14.536947 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:14.537148 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:14.537421 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:14.538493 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "931f2597e137468ea1700bfa37832616"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.543196 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:14.546751 28674 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.547610 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:21:14.547933 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "931f2597e137468ea1700bfa37832616"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.548254 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:14.566372 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:14.567701 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:14.569200 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:14.571725 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:14.571957 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.572235 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:14.572436 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.618777 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:33699
I20260709 12:21:14.618885 28736 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:33699 every 8 connection(s)
I20260709 12:21:14.624436 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:14.631951 28741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.633289 28742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.637694 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:14.638326 28737 heartbeater.cc:344] Connected to a master server at 127.16.193.126:36829
I20260709 12:21:14.638754 28737 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:14.639109 28744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.639571 28737 heartbeater.cc:507] Master 127.16.193.126:36829 requested a full tablet report, sending...
I20260709 12:21:14.640301 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:14.640583 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:14.640779 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599674640758 us; error 0 us; skew 500 ppm
I20260709 12:21:14.641427 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:14.641877 28610 ts_manager.cc:194] Registered new tserver with Master: 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699)
I20260709 12:21:14.644129 17157 webserver.cc:533] Webserver started at http://127.16.193.66:46353/ using document root <none> and password file <none>
I20260709 12:21:14.644228 28610 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57510
I20260709 12:21:14.644765 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:14.645051 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:14.645342 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:14.646598 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "d1945ee0314e465a875344905594c858"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.651204 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:14.654640 28749 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.655413 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:21:14.655726 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "d1945ee0314e465a875344905594c858"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.655992 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:14.676549 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:14.677752 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:14.679179 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:14.681389 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:14.681600 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.681845 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:14.682014 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.722735 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:45499
I20260709 12:21:14.722854 28811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:45499 every 8 connection(s)
I20260709 12:21:14.727593 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:14.734865 28815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.736498 28816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:14.738734 28818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:14.739857 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:14.740890 28812 heartbeater.cc:344] Connected to a master server at 127.16.193.126:36829
I20260709 12:21:14.741024 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:21:14.741302 28812 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:14.741407 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:14.741761 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599674741739 us; error 0 us; skew 500 ppm
I20260709 12:21:14.742226 28812 heartbeater.cc:507] Master 127.16.193.126:36829 requested a full tablet report, sending...
I20260709 12:21:14.742390 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:14.744467 28610 ts_manager.cc:194] Registered new tserver with Master: d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:14.745582 17157 webserver.cc:533] Webserver started at http://127.16.193.67:45329/ using document root <none> and password file <none>
I20260709 12:21:14.746294 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:14.746392 28610 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57512
I20260709 12:21:14.746575 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:14.746991 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:14.748551 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "8538ce0a884c4a10ae31c81b33103c07"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.753377 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:14.756696 28823 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.757544 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:14.757860 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "8538ce0a884c4a10ae31c81b33103c07"
format_stamp: "Formatted at 2026-07-09 12:21:14 on dist-test-slave-n640"
I20260709 12:21:14.758211 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:14.773030 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:14.774225 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:14.775712 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:14.777823 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:14.778048 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.778287 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:14.778460 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:14.817472 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:45905
I20260709 12:21:14.817574 28885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:45905 every 8 connection(s)
I20260709 12:21:14.832036 28886 heartbeater.cc:344] Connected to a master server at 127.16.193.126:36829
I20260709 12:21:14.832585 28886 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:14.833413 28886 heartbeater.cc:507] Master 127.16.193.126:36829 requested a full tablet report, sending...
I20260709 12:21:14.835753 28610 ts_manager.cc:194] Registered new tserver with Master: 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:14.836558 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015817537s
I20260709 12:21:14.837145 28610 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57524
I20260709 12:21:14.861907 28610 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57526:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:14.864705 28610 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:14.934760 28850 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcb1327519a4d6dba7315ab535b87b3 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:14.935861 28851 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543962a89c490a9572dc8a6ef7c792 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:14.936372 28850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcb1327519a4d6dba7315ab535b87b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.936407 28849 tablet_service.cc:1511] Processing CreateTablet for tablet 58101c530cc743ab98b421b33f16ff72 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:14.937302 28851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543962a89c490a9572dc8a6ef7c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.937891 28849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58101c530cc743ab98b421b33f16ff72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.938376 28702 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543962a89c490a9572dc8a6ef7c792 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:14.938980 28700 tablet_service.cc:1511] Processing CreateTablet for tablet 58101c530cc743ab98b421b33f16ff72 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:14.939859 28702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543962a89c490a9572dc8a6ef7c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.941987 28701 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcb1327519a4d6dba7315ab535b87b3 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:14.943342 28701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcb1327519a4d6dba7315ab535b87b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.945480 28700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58101c530cc743ab98b421b33f16ff72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.956420 28699 tablet_service.cc:1511] Processing CreateTablet for tablet 88c811c463e64eff91a0ed703c04cce9 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:14.957967 28699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88c811c463e64eff91a0ed703c04cce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.959853 28777 tablet_service.cc:1511] Processing CreateTablet for tablet 0e543962a89c490a9572dc8a6ef7c792 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:14.961318 28777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e543962a89c490a9572dc8a6ef7c792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.964892 28776 tablet_service.cc:1511] Processing CreateTablet for tablet 3dcb1327519a4d6dba7315ab535b87b3 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:14.966276 28776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dcb1327519a4d6dba7315ab535b87b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.976167 28775 tablet_service.cc:1511] Processing CreateTablet for tablet 58101c530cc743ab98b421b33f16ff72 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:14.977609 28775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58101c530cc743ab98b421b33f16ff72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.981418 28848 tablet_service.cc:1511] Processing CreateTablet for tablet 88c811c463e64eff91a0ed703c04cce9 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:14.982831 28848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88c811c463e64eff91a0ed703c04cce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.996951 28774 tablet_service.cc:1511] Processing CreateTablet for tablet 88c811c463e64eff91a0ed703c04cce9 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:14.998437 28774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88c811c463e64eff91a0ed703c04cce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.003686 28773 tablet_service.cc:1511] Processing CreateTablet for tablet 58fdc9b9c1cc431897d0f1242d1fdccf (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:15.005085 28773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58fdc9b9c1cc431897d0f1242d1fdccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.006248 28847 tablet_service.cc:1511] Processing CreateTablet for tablet 58fdc9b9c1cc431897d0f1242d1fdccf (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:15.007690 28847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58fdc9b9c1cc431897d0f1242d1fdccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.023343 28901 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.027721 28902 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.027971 28701 tablet_service.cc:1511] Processing CreateTablet for tablet 58fdc9b9c1cc431897d0f1242d1fdccf (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:15.029364 28701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58fdc9b9c1cc431897d0f1242d1fdccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.038630 28699 tablet_service.cc:1511] Processing CreateTablet for tablet 22170366e5214d469b9f37d4c7c7c30f (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:15.040061 28699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22170366e5214d469b9f37d4c7c7c30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.045552 28700 tablet_service.cc:1511] Processing CreateTablet for tablet 17e3fa379b7d4e20ad78550c233c2389 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:15.045934 28772 tablet_service.cc:1511] Processing CreateTablet for tablet 17e3fa379b7d4e20ad78550c233c2389 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:15.040014 28777 tablet_service.cc:1511] Processing CreateTablet for tablet 22170366e5214d469b9f37d4c7c7c30f (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:15.047077 28700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e3fa379b7d4e20ad78550c233c2389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.047498 28850 tablet_service.cc:1511] Processing CreateTablet for tablet 17e3fa379b7d4e20ad78550c233c2389 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:15.048821 28850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e3fa379b7d4e20ad78550c233c2389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.051775 28901 tablet_bootstrap.cc:654] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.056058 28772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e3fa379b7d4e20ad78550c233c2389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.057557 28847 tablet_service.cc:1511] Processing CreateTablet for tablet 22170366e5214d469b9f37d4c7c7c30f (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:15.058017 28777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22170366e5214d469b9f37d4c7c7c30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.061075 28698 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce98f1bed204d93ad21d02af807727c (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:15.062482 28698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce98f1bed204d93ad21d02af807727c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.064790 28776 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce98f1bed204d93ad21d02af807727c (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:15.066114 28776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce98f1bed204d93ad21d02af807727c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.071833 28903 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.058907 28847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22170366e5214d469b9f37d4c7c7c30f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.081064 28902 tablet_bootstrap.cc:654] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.084084 28848 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce98f1bed204d93ad21d02af807727c (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:15.085436 28702 tablet_service.cc:1511] Processing CreateTablet for tablet 091ceec6e2ad4b22be7737dcaf361b63 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:15.087445 28702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 091ceec6e2ad4b22be7737dcaf361b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.090415 28903 tablet_bootstrap.cc:654] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.093001 28848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce98f1bed204d93ad21d02af807727c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.100361 28772 tablet_service.cc:1511] Processing CreateTablet for tablet 091ceec6e2ad4b22be7737dcaf361b63 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:15.101476 28901 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.101605 28772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 091ceec6e2ad4b22be7737dcaf361b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.102044 28901 ts_tablet_manager.cc:1403] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.079s	user 0.006s	sys 0.005s
I20260709 12:21:15.105090 28901 raft_consensus.cc:359] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.105823 28901 raft_consensus.cc:385] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.106125 28901 raft_consensus.cc:740] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.106851 28901 consensus_queue.cc:260] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.112715 28847 tablet_service.cc:1511] Processing CreateTablet for tablet 091ceec6e2ad4b22be7737dcaf361b63 (DEFAULT_TABLE table=test-workload [id=d92c12c64ea5401d94b5f8c767f921e8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:15.114042 28847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 091ceec6e2ad4b22be7737dcaf361b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.117362 28901 ts_tablet_manager.cc:1434] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.004s
I20260709 12:21:15.118130 28737 heartbeater.cc:499] Master 127.16.193.126:36829 was elected leader, sending a full tablet report...
I20260709 12:21:15.124825 28902 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.125411 28902 ts_tablet_manager.cc:1403] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.005s
I20260709 12:21:15.127991 28903 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.128556 28903 ts_tablet_manager.cc:1403] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.002s
I20260709 12:21:15.129257 28901 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.131780 28903 raft_consensus.cc:359] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.132586 28903 raft_consensus.cc:385] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.132937 28903 raft_consensus.cc:740] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.133900 28903 consensus_queue.cc:260] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.136253 28901 tablet_bootstrap.cc:654] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.128571 28902 raft_consensus.cc:359] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.139703 28902 raft_consensus.cc:385] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.139997 28902 raft_consensus.cc:740] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.140910 28902 consensus_queue.cc:260] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.143581 28903 ts_tablet_manager.cc:1434] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.007s
I20260709 12:21:15.144392 28812 heartbeater.cc:499] Master 127.16.193.126:36829 was elected leader, sending a full tablet report...
I20260709 12:21:15.149364 28903 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.150305 28902 ts_tablet_manager.cc:1434] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 12:21:15.151815 28902 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.154772 28886 heartbeater.cc:499] Master 127.16.193.126:36829 was elected leader, sending a full tablet report...
I20260709 12:21:15.157210 28903 tablet_bootstrap.cc:654] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.160470 28902 tablet_bootstrap.cc:654] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.162750 28901 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.163452 28901 ts_tablet_manager.cc:1403] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.010s
I20260709 12:21:15.166428 28901 raft_consensus.cc:359] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.167316 28901 raft_consensus.cc:385] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.167675 28901 raft_consensus.cc:740] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.168593 28901 consensus_queue.cc:260] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.171113 28901 ts_tablet_manager.cc:1434] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:15.172132 28901 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.178645 28901 tablet_bootstrap.cc:654] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.218853 28903 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.219422 28908 raft_consensus.cc:493] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.219734 28902 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.219552 28903 ts_tablet_manager.cc:1403] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.071s	user 0.020s	sys 0.048s
I20260709 12:21:15.220180 28901 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.220367 28902 ts_tablet_manager.cc:1403] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.069s	user 0.026s	sys 0.040s
I20260709 12:21:15.220886 28901 ts_tablet_manager.cc:1403] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.030s
I20260709 12:21:15.220181 28908 raft_consensus.cc:515] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.223291 28903 raft_consensus.cc:359] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.223733 28902 raft_consensus.cc:359] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.224124 28903 raft_consensus.cc:385] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.224582 28902 raft_consensus.cc:385] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.224713 28903 raft_consensus.cc:740] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.225088 28902 raft_consensus.cc:740] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.223927 28901 raft_consensus.cc:359] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.226016 28901 raft_consensus.cc:385] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.225744 28903 consensus_queue.cc:260] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.226383 28901 raft_consensus.cc:740] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.227576 28908 leader_election.cc:290] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.228098 28902 consensus_queue.cc:260] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.229454 28901 consensus_queue.cc:260] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.231985 28902 ts_tablet_manager.cc:1434] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:21:15.233033 28902 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.237105 28906 raft_consensus.cc:493] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.237627 28906 raft_consensus.cc:515] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.238579 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.239622 28712 raft_consensus.cc:2468] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.240401 28906 leader_election.cc:290] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.242303 28901 ts_tablet_manager.cc:1434] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:21:15.243419 28901 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.253113 28901 tablet_bootstrap.cc:654] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.257654 28902 tablet_bootstrap.cc:654] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.280263 28750 leader_election.cc:304] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.281665 28908 raft_consensus.cc:2804] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.282155 28908 raft_consensus.cc:493] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.282555 28908 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.298502 28901 tablet_bootstrap.cc:492] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.299173 28901 ts_tablet_manager.cc:1403] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.019s
I20260709 12:21:15.302017 28908 raft_consensus.cc:515] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.304419 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.304867 28908 leader_election.cc:290] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.302290 28901 raft_consensus.cc:359] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.305492 28901 raft_consensus.cc:385] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.305841 28901 raft_consensus.cc:740] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.306442 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.307031 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.306699 28901 consensus_queue.cc:260] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.307402 28712 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.309612 28901 ts_tablet_manager.cc:1434] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
W20260709 12:21:15.310544 28751 leader_election.cc:343] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:15.318595 28751 leader_election.cc:343] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.316985 28902 tablet_bootstrap.cc:492] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.320461 28902 ts_tablet_manager.cc:1403] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.012s
I20260709 12:21:15.321066 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:15.310729 28901 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.321973 28787 raft_consensus.cc:2393] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 931f2597e137468ea1700bfa37832616 in current term 1: Already voted for candidate d1945ee0314e465a875344905594c858 in this term.
I20260709 12:21:15.319175 28903 ts_tablet_manager.cc:1434] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.099s	user 0.052s	sys 0.044s
I20260709 12:21:15.324113 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.325181 28903 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.326408 28712 raft_consensus.cc:2468] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.323562 28902 raft_consensus.cc:359] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.327631 28902 raft_consensus.cc:385] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.327997 28902 raft_consensus.cc:740] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.328998 28901 tablet_bootstrap.cc:654] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.328790 28902 consensus_queue.cc:260] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.329788 28861 raft_consensus.cc:2468] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:15.331977 28915 raft_consensus.cc:493] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.332576 28915 raft_consensus.cc:515] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.333181 28750 leader_election.cc:304] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:15.334411 28908 raft_consensus.cc:2804] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.334802 28915 leader_election.cc:290] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905), 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699)
I20260709 12:21:15.335767 28903 tablet_bootstrap.cc:654] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.337122 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.338066 28860 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.339859 28751 leader_election.cc:304] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.341606 28676 leader_election.cc:304] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: d1945ee0314e465a875344905594c858
I20260709 12:21:15.342841 28906 raft_consensus.cc:2764] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:21:15.344551 28915 raft_consensus.cc:2804] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.345109 28915 raft_consensus.cc:493] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.345523 28915 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.347820 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.348894 28712 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.357378 28902 ts_tablet_manager.cc:1434] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:21:15.365593 28915 raft_consensus.cc:515] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.368000 28902 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.373070 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.373493 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.374039 28861 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.374365 28712 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.380052 28915 leader_election.cc:290] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905), 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699)
I20260709 12:21:15.381500 28906 raft_consensus.cc:493] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.382215 28906 raft_consensus.cc:515] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.403450 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
W20260709 12:21:15.406659 28678 leader_election.cc:343] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.407824 28902 tablet_bootstrap.cc:654] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.409986 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.410908 28861 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.411062 28860 raft_consensus.cc:2468] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:15.411729 28906 leader_election.cc:290] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.413609 28676 leader_election.cc:304] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: d1945ee0314e465a875344905594c858
I20260709 12:21:15.414548 28906 raft_consensus.cc:2804] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.414984 28906 raft_consensus.cc:493] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.411856 28908 raft_consensus.cc:697] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:15.415427 28906 raft_consensus.cc:3060] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.415608 28751 leader_election.cc:304] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.416071 28901 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.416661 28901 ts_tablet_manager.cc:1403] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.106s	user 0.052s	sys 0.036s
I20260709 12:21:15.416363 28908 consensus_queue.cc:237] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.416728 28921 raft_consensus.cc:2804] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.417430 28921 raft_consensus.cc:697] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:15.418294 28921 consensus_queue.cc:237] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:15.419482 28903 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.420007 28903 ts_tablet_manager.cc:1403] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.095s	user 0.024s	sys 0.056s
I20260709 12:21:15.420094 28901 raft_consensus.cc:359] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.420878 28901 raft_consensus.cc:385] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.421185 28901 raft_consensus.cc:740] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.422024 28901 consensus_queue.cc:260] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.425113 28906 raft_consensus.cc:515] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.430609 28906 leader_election.cc:290] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.427591 28712 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.425578 28903 raft_consensus.cc:359] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.431784 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:15.432238 28903 raft_consensus.cc:385] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.432742 28903 raft_consensus.cc:740] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.433563 28903 consensus_queue.cc:260] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
W20260709 12:21:15.433774 28678 leader_election.cc:343] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.435962 28903 ts_tablet_manager.cc:1434] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:21:15.437533 28903 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.436369 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.439859 28861 raft_consensus.cc:3060] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.443554 28901 ts_tablet_manager.cc:1434] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.005s
I20260709 12:21:15.444761 28901 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.444201 28608 catalog_manager.cc:5697] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:15.449980 28658 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:15.450014 28861 raft_consensus.cc:2468] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:15.451503 28676 leader_election.cc:304] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: d1945ee0314e465a875344905594c858
I20260709 12:21:15.452291 28902 tablet_bootstrap.cc:492] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.452881 28902 ts_tablet_manager.cc:1403] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.004s
I20260709 12:21:15.454056 28926 raft_consensus.cc:2804] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.454101 28901 tablet_bootstrap.cc:654] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.455570 28926 raft_consensus.cc:697] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:15.456331 28903 tablet_bootstrap.cc:654] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.456487 28926 consensus_queue.cc:237] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.457751 28902 raft_consensus.cc:359] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.467054 28902 raft_consensus.cc:385] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.467943 28902 raft_consensus.cc:740] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.469306 28902 consensus_queue.cc:260] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.472127 28902 ts_tablet_manager.cc:1434] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:21:15.473174 28902 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.476044 28610 catalog_manager.cc:5697] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.481879 28902 tablet_bootstrap.cc:654] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.498621 28901 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.499122 28901 ts_tablet_manager.cc:1403] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.030s
I20260709 12:21:15.500869 28903 tablet_bootstrap.cc:492] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.501497 28903 ts_tablet_manager.cc:1403] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.064s	user 0.024s	sys 0.027s
I20260709 12:21:15.502398 28901 raft_consensus.cc:359] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.503217 28901 raft_consensus.cc:385] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.503603 28901 raft_consensus.cc:740] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.504397 28901 consensus_queue.cc:260] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.504833 28903 raft_consensus.cc:359] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.505751 28903 raft_consensus.cc:385] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.506095 28903 raft_consensus.cc:740] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.506124 28902 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.506983 28901 ts_tablet_manager.cc:1434] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:21:15.507038 28902 ts_tablet_manager.cc:1403] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.002s
I20260709 12:21:15.507212 28903 consensus_queue.cc:260] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.508220 28901 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.510193 28903 ts_tablet_manager.cc:1434] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:15.511286 28903 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.512123 28902 raft_consensus.cc:359] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.513144 28902 raft_consensus.cc:385] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.513511 28902 raft_consensus.cc:740] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.517316 28901 tablet_bootstrap.cc:654] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.517135 28902 consensus_queue.cc:260] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.519330 28903 tablet_bootstrap.cc:654] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.529078 28608 catalog_manager.cc:5697] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.540836 28902 ts_tablet_manager.cc:1434] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:21:15.542153 28902 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.543536 28901 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.544281 28901 ts_tablet_manager.cc:1403] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.015s
I20260709 12:21:15.548317 28901 raft_consensus.cc:359] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.549263 28901 raft_consensus.cc:385] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.549640 28901 raft_consensus.cc:740] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.551050 28902 tablet_bootstrap.cc:654] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.550796 28901 consensus_queue.cc:260] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.553154 28903 tablet_bootstrap.cc:492] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.553704 28903 ts_tablet_manager.cc:1403] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.020s
I20260709 12:21:15.556866 28926 raft_consensus.cc:493] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.557624 28926 raft_consensus.cc:515] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.557763 28903 raft_consensus.cc:359] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.558687 28903 raft_consensus.cc:385] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.559001 28903 raft_consensus.cc:740] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.560048 28901 ts_tablet_manager.cc:1434] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:21:15.559975 28903 consensus_queue.cc:260] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.561264 28901 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.561203 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
W20260709 12:21:15.563823 28676 leader_election.cc:343] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.564388 28903 ts_tablet_manager.cc:1434] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:21:15.565469 28903 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.565537 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
W20260709 12:21:15.567516 28678 leader_election.cc:343] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.567989 28678 leader_election.cc:304] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 931f2597e137468ea1700bfa37832616; no voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858
I20260709 12:21:15.569126 28901 tablet_bootstrap.cc:654] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.572628 28903 tablet_bootstrap.cc:654] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.569077 28906 raft_consensus.cc:2749] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:15.578003 28902 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.578655 28902 ts_tablet_manager.cc:1403] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.002s
I20260709 12:21:15.579420 28921 raft_consensus.cc:493] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.580111 28921 raft_consensus.cc:515] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.582401 28902 raft_consensus.cc:359] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.583250 28902 raft_consensus.cc:385] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.583523 28921 leader_election.cc:290] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.584383 28903 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.584900 28903 ts_tablet_manager.cc:1403] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 12:21:15.586266 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.584630 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58101c530cc743ab98b421b33f16ff72" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.588188 28903 raft_consensus.cc:359] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.588945 28903 raft_consensus.cc:385] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.589166 28861 raft_consensus.cc:2393] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1945ee0314e465a875344905594c858 in current term 1: Already voted for candidate 931f2597e137468ea1700bfa37832616 in this term.
I20260709 12:21:15.589262 28903 raft_consensus.cc:740] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.590427 28903 consensus_queue.cc:260] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.590991 28751 leader_election.cc:304] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616
I20260709 12:21:15.592707 28926 leader_election.cc:290] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.592808 28903 ts_tablet_manager.cc:1434] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:21:15.592919 28908 raft_consensus.cc:3060] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.593626 28926 raft_consensus.cc:493] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.594009 28903 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.594089 28926 raft_consensus.cc:515] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.597191 28926 leader_election.cc:290] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.583542 28902 raft_consensus.cc:740] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.598574 28902 consensus_queue.cc:260] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.600453 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:15.601388 28903 tablet_bootstrap.cc:654] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.601356 28787 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:15.603070 28902 ts_tablet_manager.cc:1434] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:21:15.604182 28902 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.607659 28915 raft_consensus.cc:493] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.608320 28915 raft_consensus.cc:515] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.613103 28678 leader_election.cc:304] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.613547 28902 tablet_bootstrap.cc:654] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.613777 28903 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.613858 28901 tablet_bootstrap.cc:492] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.614037 28906 raft_consensus.cc:2804] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.614509 28903 ts_tablet_manager.cc:1403] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 12:21:15.614735 28901 ts_tablet_manager.cc:1403] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.054s	user 0.025s	sys 0.026s
I20260709 12:21:15.615864 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.617573 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.618233 28712 raft_consensus.cc:2468] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.618270 28903 raft_consensus.cc:359] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.619355 28903 raft_consensus.cc:385] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.619750 28903 raft_consensus.cc:740] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.620105 28901 raft_consensus.cc:359] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.620939 28901 raft_consensus.cc:385] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:15.621169 28751 leader_election.cc:343] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:15.621471 28901 raft_consensus.cc:740] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.620790 28903 consensus_queue.cc:260] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.622362 28750 leader_election.cc:304] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:15.622366 28901 consensus_queue.cc:260] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.624356 28903 ts_tablet_manager.cc:1434] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.005s
I20260709 12:21:15.625685 28903 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.627266 28901 ts_tablet_manager.cc:1434] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:21:15.628305 28901 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:15.611133 28915 leader_election.cc:290] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.634274 28901 tablet_bootstrap.cc:654] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.634632 28915 raft_consensus.cc:2804] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.635012 28910 raft_consensus.cc:493] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.635656 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.635751 28910 raft_consensus.cc:515] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.636528 28861 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:15.641356 28910 leader_election.cc:290] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.614624 28906 raft_consensus.cc:493] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.642771 28906 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.635092 28915 raft_consensus.cc:493] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.643608 28915 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.654204 28902 tablet_bootstrap.cc:492] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.654884 28902 ts_tablet_manager.cc:1403] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.026s
I20260709 12:21:15.658519 28902 raft_consensus.cc:359] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.659193 28902 raft_consensus.cc:385] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.659516 28902 raft_consensus.cc:740] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.660346 28902 consensus_queue.cc:260] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.663070 28902 ts_tablet_manager.cc:1434] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:21:15.664148 28902 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.664045 28915 raft_consensus.cc:515] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.666358 28915 leader_election.cc:290] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.673693 28908 raft_consensus.cc:2749] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:15.674495 28908 raft_consensus.cc:493] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.674947 28908 raft_consensus.cc:515] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.677263 28902 tablet_bootstrap.cc:654] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.678443 28906 raft_consensus.cc:515] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.688515 28901 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:15.689112 28901 ts_tablet_manager.cc:1403] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.033s
I20260709 12:21:15.691359 28906 leader_election.cc:290] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
W20260709 12:21:15.691031 28750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.16.193.65:33699, user_credentials={real_user=slave}} blocked reactor thread for 68761.2us
I20260709 12:21:15.691591 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.692273 28861 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.687896 28908 leader_election.cc:290] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.695611 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.696236 28860 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.701871 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.702625 28711 raft_consensus.cc:2468] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.703972 28824 leader_election.cc:304] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:15.704053 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:15.704708 28787 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.704983 28910 raft_consensus.cc:2804] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.704903 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.705557 28910 raft_consensus.cc:493] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.705626 28901 raft_consensus.cc:359] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.706467 28901 raft_consensus.cc:385] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.706873 28901 raft_consensus.cc:740] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.707713 28901 consensus_queue.cc:260] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.708287 28903 tablet_bootstrap.cc:654] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.709518 28786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:15.712208 28786 raft_consensus.cc:2468] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.706053 28910 raft_consensus.cc:3060] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.709982 28901 ts_tablet_manager.cc:1434] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:21:15.702749 28902 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.714686 28902 ts_tablet_manager.cc:1403] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.017s
I20260709 12:21:15.714999 28787 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:15.716135 28678 leader_election.cc:304] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.716421 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.716988 28906 raft_consensus.cc:2804] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.717020 28712 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.718497 28902 raft_consensus.cc:359] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.719504 28902 raft_consensus.cc:385] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.719883 28902 raft_consensus.cc:740] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.718407 28906 raft_consensus.cc:697] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:15.721705 28902 consensus_queue.cc:260] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.722201 28906 consensus_queue.cc:237] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.727101 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.727798 28711 raft_consensus.cc:2468] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.728960 28750 leader_election.cc:304] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.729888 28908 raft_consensus.cc:2804] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.730252 28908 raft_consensus.cc:493] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.730574 28908 raft_consensus.cc:3060] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.735554 28860 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:15.739409 28908 raft_consensus.cc:515] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.745379 28903 tablet_bootstrap.cc:492] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.745589 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.746047 28903 ts_tablet_manager.cc:1403] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.009s
I20260709 12:21:15.746192 28711 raft_consensus.cc:3060] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.749056 28903 raft_consensus.cc:359] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.749722 28903 raft_consensus.cc:385] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.725646 28861 raft_consensus.cc:2468] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.750023 28903 raft_consensus.cc:740] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.751358 28751 leader_election.cc:304] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.753500 28946 raft_consensus.cc:2804] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.754151 28946 raft_consensus.cc:697] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:15.755043 28946 consensus_queue.cc:237] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.755506 28903 consensus_queue.cc:260] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.756085 28711 raft_consensus.cc:2468] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.740136 28910 raft_consensus.cc:515] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.742666 28712 raft_consensus.cc:2468] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.751904 28908 leader_election.cc:290] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.759792 28861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.759815 28750 leader_election.cc:304] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.760911 28908 raft_consensus.cc:2804] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.761555 28908 raft_consensus.cc:697] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:15.762907 28903 ts_tablet_manager.cc:1434] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:21:15.764025 28903 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:15.764569 28902 ts_tablet_manager.cc:1434] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.050s	user 0.015s	sys 0.024s
I20260709 12:21:15.765017 28908 consensus_queue.cc:237] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.765545 28902 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:15.770123 28859 raft_consensus.cc:2393] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1945ee0314e465a875344905594c858 in current term 1: Already voted for candidate 8538ce0a884c4a10ae31c81b33103c07 in this term.
I20260709 12:21:15.770421 28608 catalog_manager.cc:5697] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.773901 28910 leader_election.cc:290] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.774659 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.774755 28910 raft_consensus.cc:493] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.775576 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58fdc9b9c1cc431897d0f1242d1fdccf" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:15.776290 28910 raft_consensus.cc:515] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.775588 28711 raft_consensus.cc:2393] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8538ce0a884c4a10ae31c81b33103c07 in current term 1: Already voted for candidate d1945ee0314e465a875344905594c858 in this term.
I20260709 12:21:15.782431 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.782995 28712 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.783475 28824 leader_election.cc:304] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07; no voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858
I20260709 12:21:15.787155 28824 leader_election.cc:304] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:15.788122 28950 raft_consensus.cc:2749] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:15.788797 28950 raft_consensus.cc:2804] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.789104 28910 leader_election.cc:290] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.790174 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:15.790863 28787 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.791601 28903 tablet_bootstrap.cc:654] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.791985 28902 tablet_bootstrap.cc:654] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:15.793370 28610 catalog_manager.cc:5697] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.796656 28950 raft_consensus.cc:493] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.800146 28950 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.806520 28903 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:15.807034 28903 ts_tablet_manager.cc:1403] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.001s
I20260709 12:21:15.809880 28903 raft_consensus.cc:359] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.810493 28903 raft_consensus.cc:385] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.810765 28903 raft_consensus.cc:740] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.812376 28950 raft_consensus.cc:515] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.812840 28903 consensus_queue.cc:260] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.814180 28902 tablet_bootstrap.cc:492] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:15.814677 28902 ts_tablet_manager.cc:1403] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.020s
I20260709 12:21:15.816792 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:15.816067 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.818317 28902 raft_consensus.cc:359] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.818626 28787 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.818768 28712 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.819036 28902 raft_consensus.cc:385] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:15.819469 28902 raft_consensus.cc:740] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:15.818547 28608 catalog_manager.cc:5697] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.820207 28902 consensus_queue.cc:260] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.823648 28902 ts_tablet_manager.cc:1434] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:21:15.825881 28712 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:15.827033 28824 leader_election.cc:304] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:15.827173 28903 ts_tablet_manager.cc:1434] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.002s
I20260709 12:21:15.827085 28787 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:15.828012 28910 raft_consensus.cc:2804] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.815721 28950 leader_election.cc:290] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.828567 28910 raft_consensus.cc:697] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:15.829277 28910 consensus_queue.cc:237] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.833923 28946 consensus_queue.cc:1048] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:15.841867 28610 catalog_manager.cc:5697] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:15.845901 28860 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.855554 28908 consensus_queue.cc:1048] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:15.874169 28910 raft_consensus.cc:493] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.874562 28910 raft_consensus.cc:515] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.876418 28910 leader_election.cc:290] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.877260 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.877497 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:15.878011 28712 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.878085 28787 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:15.879159 28824 leader_election.cc:304] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:15.879876 28910 raft_consensus.cc:2804] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.880247 28910 raft_consensus.cc:493] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.880563 28910 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.885320 28910 raft_consensus.cc:515] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.886847 28910 leader_election.cc:290] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:15.887681 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.887864 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:15.888195 28712 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.888492 28787 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.893354 28787 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:15.894258 28712 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:15.894460 28827 leader_election.cc:304] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.895305 28910 raft_consensus.cc:2804] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.895747 28910 raft_consensus.cc:697] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:15.896517 28910 consensus_queue.cc:237] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.903513 28610 catalog_manager.cc:5697] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:15.933992 28906 consensus_queue.cc:1048] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:15.942134 28908 raft_consensus.cc:493] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:15.944000 28908 raft_consensus.cc:515] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.945842 28965 consensus_queue.cc:1048] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:15.948038 28908 leader_election.cc:290] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.948226 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:15.948905 28860 raft_consensus.cc:2468] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.948405 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:15.950022 28712 raft_consensus.cc:2468] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:15.950037 28751 leader_election.cc:304] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.951100 28908 raft_consensus.cc:2804] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:15.951560 28908 raft_consensus.cc:493] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:15.951947 28908 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.958828 28908 raft_consensus.cc:515] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.961002 28908 leader_election.cc:290] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:15.961305 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:15.961697 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:15.961911 28712 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.962388 28860 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:15.968892 28860 raft_consensus.cc:2468] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.969053 28712 raft_consensus.cc:2468] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:15.969978 28751 leader_election.cc:304] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:15.970795 28908 raft_consensus.cc:2804] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:15.971256 28908 raft_consensus.cc:697] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:15.971964 28908 consensus_queue.cc:237] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:15.979380 28608 catalog_manager.cc:5697] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.997299 28908 consensus_queue.cc:1048] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.007957 28946 consensus_queue.cc:1048] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.098222 28787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:52424
I20260709 12:21:16.098752 28787 raft_consensus.cc:606] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Received request to transfer leadership to TS 931f2597e137468ea1700bfa37832616
I20260709 12:21:16.103586 28787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:52424
I20260709 12:21:16.104074 28787 raft_consensus.cc:606] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Received request to transfer leadership to TS 931f2597e137468ea1700bfa37832616
I20260709 12:21:16.107442 28965 consensus_queue.cc:1048] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.120268 28906 consensus_queue.cc:1048] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.139930 28946 consensus_queue.cc:1048] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.144636 28859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
mode: GRACEFUL
new_leader_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:39246
I20260709 12:21:16.145326 28859 raft_consensus.cc:606] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Received request to transfer leadership to TS d1945ee0314e465a875344905594c858
I20260709 12:21:16.155988 28915 consensus_queue.cc:1048] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.156692 28859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
mode: GRACEFUL
new_leader_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:39246
I20260709 12:21:16.157364 28859 raft_consensus.cc:606] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Received request to transfer leadership to TS d1945ee0314e465a875344905594c858
I20260709 12:21:16.163671 28787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:52424
I20260709 12:21:16.164289 28787 raft_consensus.cc:606] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Received request to transfer leadership to TS 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:16.181473 28915 consensus_queue.cc:1048] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.192744 28915 consensus_queue.cc:1048] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.271569 28910 consensus_queue.cc:1048] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.282007 28952 consensus_queue.cc:1048] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.289369 28950 raft_consensus.cc:993] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: : Instructing follower d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:16.289850 28952 raft_consensus.cc:1081] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Signalling peer d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:16.291330 28787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389"
dest_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:52418
I20260709 12:21:16.291893 28787 raft_consensus.cc:493] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:16.292282 28787 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.297235 28787 raft_consensus.cc:515] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.298945 28787 leader_election.cc:290] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:16.300083 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:16.300706 28859 raft_consensus.cc:3055] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:16.300981 28859 raft_consensus.cc:740] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:16.300045 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e3fa379b7d4e20ad78550c233c2389" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:16.301712 28712 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.301990 28859 consensus_queue.cc:260] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.303265 28859 raft_consensus.cc:3060] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.307279 28712 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:16.308413 28750 leader_election.cc:304] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:16.308481 28859 raft_consensus.cc:2468] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:16.309183 28915 raft_consensus.cc:2804] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.309741 28915 raft_consensus.cc:697] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:16.310482 28915 consensus_queue.cc:237] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.317199 28952 consensus_queue.cc:1048] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.317368 28607 catalog_manager.cc:5697] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 1 to 2, leader changed from 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67) to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.331614 28952 consensus_queue.cc:1048] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.339154 28910 raft_consensus.cc:993] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: : Instructing follower d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:16.339674 28952 raft_consensus.cc:1081] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Signalling peer d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:16.341349 28787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c"
dest_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:52418
I20260709 12:21:16.341821 28787 raft_consensus.cc:493] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:16.342118 28787 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.347084 28787 raft_consensus.cc:515] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.349721 28787 leader_election.cc:290] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:16.349923 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:16.349996 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce98f1bed204d93ad21d02af807727c" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:16.350677 28712 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.350759 28859 raft_consensus.cc:3055] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:16.351266 28859 raft_consensus.cc:740] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:16.352065 28859 consensus_queue.cc:260] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.353027 28859 raft_consensus.cc:3060] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.356709 28712 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:16.357708 28750 leader_election.cc:304] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:16.358438 28915 raft_consensus.cc:2804] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.358655 28859 raft_consensus.cc:2468] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:16.358958 28915 raft_consensus.cc:697] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:16.359882 28915 consensus_queue.cc:237] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.366097 28607 catalog_manager.cc:5697] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 1 to 2, leader changed from 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67) to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.409511 28977 raft_consensus.cc:993] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: : Instructing follower 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:16.409991 28908 raft_consensus.cc:1081] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Signalling peer 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:16.411419 28712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3"
dest_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:42328
I20260709 12:21:16.411924 28712 raft_consensus.cc:493] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:16.412205 28712 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.417241 28712 raft_consensus.cc:515] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.418780 28712 leader_election.cc:290] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:16.419595 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:16.419823 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dcb1327519a4d6dba7315ab535b87b3" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:16.420168 28787 raft_consensus.cc:3055] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:16.420432 28860 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.420547 28787 raft_consensus.cc:740] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:16.421326 28787 consensus_queue.cc:260] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.422276 28787 raft_consensus.cc:3060] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.425745 28860 raft_consensus.cc:2468] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:16.426775 28676 leader_election.cc:304] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:16.427601 28965 raft_consensus.cc:2804] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.427757 28787 raft_consensus.cc:2468] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:16.428001 28965 raft_consensus.cc:697] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:16.428905 28965 consensus_queue.cc:237] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.434878 28608 catalog_manager.cc:5697] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 1 to 2, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.519596 28908 consensus_queue.cc:1048] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.527594 28908 raft_consensus.cc:993] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: : Instructing follower 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:16.528035 28908 raft_consensus.cc:1081] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Signalling peer 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:16.530031 28859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:39236
I20260709 12:21:16.530587 28859 raft_consensus.cc:493] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:16.530992 28859 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.536564 28908 consensus_queue.cc:1048] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:16.540145 28859 raft_consensus.cc:515] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.541409 28860 raft_consensus.cc:1240] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Rejecting Update request from peer d1945ee0314e465a875344905594c858 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:16.542712 28859 leader_election.cc:290] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:16.543359 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:16.546173 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "091ceec6e2ad4b22be7737dcaf361b63" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:16.546675 28908 consensus_queue.cc:1059] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:16.549005 28787 raft_consensus.cc:3055] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:16.549348 28787 raft_consensus.cc:740] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:16.550215 28787 consensus_queue.cc:260] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
W20260709 12:21:16.551445 28915 consensus_queue.cc:1175] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "931f2597e137468ea1700bfa37832616" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:21:16.551656 28787 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.558022 28787 raft_consensus.cc:2468] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 2.
W20260709 12:21:16.558499 28908 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:21:16.559183 28827 leader_election.cc:304] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 931f2597e137468ea1700bfa37832616
I20260709 12:21:16.560133 28952 raft_consensus.cc:2804] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.560571 28952 raft_consensus.cc:697] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:16.561187 28952 consensus_queue.cc:237] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:16.566869 28610 catalog_manager.cc:5697] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 1 to 2, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:16.615041 28977 raft_consensus.cc:993] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: : Instructing follower 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:16.615561 28908 raft_consensus.cc:1081] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Signalling peer 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:16.617028 28712 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792"
dest_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:42328
I20260709 12:21:16.617540 28712 raft_consensus.cc:493] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:16.617928 28712 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.622995 28712 raft_consensus.cc:515] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:16.624577 28712 leader_election.cc:290] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Requested vote from peers 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:16.625579 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:16.625656 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e543962a89c490a9572dc8a6ef7c792" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:16.626263 28860 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.626332 28787 raft_consensus.cc:3055] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:16.626765 28787 raft_consensus.cc:740] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:16.627585 28787 consensus_queue.cc:260] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:16.628508 28787 raft_consensus.cc:3060] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.632261 28860 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:16.633764 28676 leader_election.cc:304] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:16.634554 28965 raft_consensus.cc:2804] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.635811 28965 raft_consensus.cc:697] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:16.636080 28787 raft_consensus.cc:2468] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:16.636655 28965 consensus_queue.cc:237] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } }
I20260709 12:21:16.643203 28610 catalog_manager.cc:5697] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 1 to 2, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:16.753223 28860 raft_consensus.cc:1275] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.754487 28908 consensus_queue.cc:1048] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.763540 28712 raft_consensus.cc:1275] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.765355 28946 consensus_queue.cc:1048] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.847674 28787 raft_consensus.cc:1275] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.849113 28965 consensus_queue.cc:1048] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.858553 28860 raft_consensus.cc:1275] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.860373 28926 consensus_queue.cc:1048] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.883502 28860 raft_consensus.cc:1275] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.884793 28946 consensus_queue.cc:1048] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.892804 28712 raft_consensus.cc:1275] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.894299 28946 consensus_queue.cc:1048] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.986100 28712 raft_consensus.cc:3060] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.991231 28712 raft_consensus.cc:1275] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.992439 28952 consensus_queue.cc:1048] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:17.001240 28787 raft_consensus.cc:1275] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.002766 28986 consensus_queue.cc:1048] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:17.152078 28860 raft_consensus.cc:1275] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.153528 28906 consensus_queue.cc:1048] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:17.164405 28787 raft_consensus.cc:1275] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.165828 28926 consensus_queue.cc:1048] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.191803 28607 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57540:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:19.194257 28607 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:19.245052 28698 tablet_service.cc:1511] Processing CreateTablet for tablet 02d033d6a3fe4eb19439c707068a7b10 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:19.246384 28698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d033d6a3fe4eb19439c707068a7b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.246044 28776 tablet_service.cc:1511] Processing CreateTablet for tablet 02d033d6a3fe4eb19439c707068a7b10 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:19.247457 28776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d033d6a3fe4eb19439c707068a7b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.248353 28847 tablet_service.cc:1511] Processing CreateTablet for tablet 02d033d6a3fe4eb19439c707068a7b10 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:19.249665 28847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d033d6a3fe4eb19439c707068a7b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.250269 28700 tablet_service.cc:1511] Processing CreateTablet for tablet dd4c0144eb8346fa8f1080616ea29109 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:19.251540 28700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4c0144eb8346fa8f1080616ea29109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.254541 28848 tablet_service.cc:1511] Processing CreateTablet for tablet dd4c0144eb8346fa8f1080616ea29109 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:19.255852 28702 tablet_service.cc:1511] Processing CreateTablet for tablet d2889961de7345ee9e307539e41a5a89 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:19.257130 28702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2889961de7345ee9e307539e41a5a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.256829 28772 tablet_service.cc:1511] Processing CreateTablet for tablet dd4c0144eb8346fa8f1080616ea29109 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:19.258065 28772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4c0144eb8346fa8f1080616ea29109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.260363 28777 tablet_service.cc:1511] Processing CreateTablet for tablet d2889961de7345ee9e307539e41a5a89 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:19.260687 28774 tablet_service.cc:1511] Processing CreateTablet for tablet 488a5535ca71476db152cd1b9dfe5347 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:19.261632 28777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2889961de7345ee9e307539e41a5a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.262742 28699 tablet_service.cc:1511] Processing CreateTablet for tablet 488a5535ca71476db152cd1b9dfe5347 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:19.264035 28699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488a5535ca71476db152cd1b9dfe5347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.264333 28850 tablet_service.cc:1511] Processing CreateTablet for tablet d2889961de7345ee9e307539e41a5a89 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:19.255858 28848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd4c0144eb8346fa8f1080616ea29109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.265923 28774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488a5535ca71476db152cd1b9dfe5347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.267721 28850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2889961de7345ee9e307539e41a5a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.282605 28849 tablet_service.cc:1511] Processing CreateTablet for tablet 488a5535ca71476db152cd1b9dfe5347 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:19.284093 28849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488a5535ca71476db152cd1b9dfe5347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.290350 29003 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.299193 28702 tablet_service.cc:1511] Processing CreateTablet for tablet 1b575713ccdc48bfb6616b5e1afc0c60 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:19.300782 28702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b575713ccdc48bfb6616b5e1afc0c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.311527 29003 tablet_bootstrap.cc:654] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.311213 28775 tablet_service.cc:1511] Processing CreateTablet for tablet 1b575713ccdc48bfb6616b5e1afc0c60 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:19.312999 28775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b575713ccdc48bfb6616b5e1afc0c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.326465 29004 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.327302 28851 tablet_service.cc:1511] Processing CreateTablet for tablet 1b575713ccdc48bfb6616b5e1afc0c60 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:19.328733 28851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b575713ccdc48bfb6616b5e1afc0c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.329377 28773 tablet_service.cc:1511] Processing CreateTablet for tablet fb77d6947fd44daf919b6e8af52861e7 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:19.330717 28773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb77d6947fd44daf919b6e8af52861e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.333279 28701 tablet_service.cc:1511] Processing CreateTablet for tablet fb77d6947fd44daf919b6e8af52861e7 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:19.334636 28701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb77d6947fd44daf919b6e8af52861e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.338791 28697 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fefc8bf46465ea8fd628770cd4ba4 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:19.339493 28846 tablet_service.cc:1511] Processing CreateTablet for tablet fb77d6947fd44daf919b6e8af52861e7 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:19.340190 28697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fefc8bf46465ea8fd628770cd4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.340296 28771 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fefc8bf46465ea8fd628770cd4ba4 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:19.340945 28846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb77d6947fd44daf919b6e8af52861e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.341720 28771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fefc8bf46465ea8fd628770cd4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.373476 29004 tablet_bootstrap.cc:654] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.402551 29005 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.404836 28850 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fefc8bf46465ea8fd628770cd4ba4 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:19.406360 28850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fefc8bf46465ea8fd628770cd4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.409632 29005 tablet_bootstrap.cc:654] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.413525 29003 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.424343 28700 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1531185db849d8bcb282065ffaa5c6 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:19.429222 28776 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1531185db849d8bcb282065ffaa5c6 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:19.430480 28700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1531185db849d8bcb282065ffaa5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.430409 28851 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1531185db849d8bcb282065ffaa5c6 (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:19.431687 28851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1531185db849d8bcb282065ffaa5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.430759 28776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1531185db849d8bcb282065ffaa5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.439391 28702 tablet_service.cc:1511] Processing CreateTablet for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:19.440718 29003 ts_tablet_manager.cc:1403] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.151s	user 0.033s	sys 0.019s
I20260709 12:21:19.440865 28702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.453050 29003 raft_consensus.cc:359] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.453893 29003 raft_consensus.cc:385] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.454280 29003 raft_consensus.cc:740] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.454001 28848 tablet_service.cc:1511] Processing CreateTablet for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:19.458159 28848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.458981 28771 tablet_service.cc:1511] Processing CreateTablet for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed (DEFAULT_TABLE table=soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:19.476227 28771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489cf4ccbbeb4458ab1d3ca94f1f89ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:19.455145 29003 consensus_queue.cc:260] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
W20260709 12:21:19.486517 28658 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:19.511660 29005 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.512920 29005 ts_tablet_manager.cc:1403] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.021s
I20260709 12:21:19.516628 29005 raft_consensus.cc:359] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.517401 29005 raft_consensus.cc:385] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.517714 29005 raft_consensus.cc:740] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.518883 29004 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.519448 29004 ts_tablet_manager.cc:1403] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.193s	user 0.027s	sys 0.012s
I20260709 12:21:19.520069 29003 ts_tablet_manager.cc:1434] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.004s
I20260709 12:21:19.521190 29003 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.522464 29004 raft_consensus.cc:359] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.523259 29004 raft_consensus.cc:385] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.523576 29004 raft_consensus.cc:740] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.524349 29004 consensus_queue.cc:260] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.525566 29005 consensus_queue.cc:260] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.527499 29005 ts_tablet_manager.cc:1434] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:21:19.528539 29005 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.535539 29005 tablet_bootstrap.cc:654] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.540134 29003 tablet_bootstrap.cc:654] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.546063 29004 ts_tablet_manager.cc:1434] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:21:19.548921 29005 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.549443 29005 ts_tablet_manager.cc:1403] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:21:19.553006 29005 raft_consensus.cc:359] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.553747 29005 raft_consensus.cc:385] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.554023 29005 raft_consensus.cc:740] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
W20260709 12:21:19.539870 28594 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.16.193.65:33699, user_credentials={real_user=slave}} blocked reactor thread for 50794.7us
I20260709 12:21:19.554742 29004 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.557801 29005 consensus_queue.cc:260] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.559427 29003 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.559942 29003 ts_tablet_manager.cc:1403] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.000s
I20260709 12:21:19.560899 29005 ts_tablet_manager.cc:1434] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:19.561971 29005 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.562747 29004 tablet_bootstrap.cc:654] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.562948 29003 raft_consensus.cc:359] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.563719 29003 raft_consensus.cc:385] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.564004 29003 raft_consensus.cc:740] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.564718 29003 consensus_queue.cc:260] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.566967 29003 ts_tablet_manager.cc:1434] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:19.568154 29003 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.571043 29005 tablet_bootstrap.cc:654] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.574429 29003 tablet_bootstrap.cc:654] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.589419 29003 tablet_bootstrap.cc:492] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.589972 29003 ts_tablet_manager.cc:1403] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.012s
I20260709 12:21:19.593015 29003 raft_consensus.cc:359] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.593799 29003 raft_consensus.cc:385] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.594127 29003 raft_consensus.cc:740] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.594907 29003 consensus_queue.cc:260] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.595682 29005 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.596056 29004 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.596251 29005 ts_tablet_manager.cc:1403] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.021s
I20260709 12:21:19.596632 29004 ts_tablet_manager.cc:1403] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.043s	user 0.029s	sys 0.005s
I20260709 12:21:19.599571 29005 raft_consensus.cc:359] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.600018 29004 raft_consensus.cc:359] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.600446 29003 ts_tablet_manager.cc:1434] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:21:19.600430 29005 raft_consensus.cc:385] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.600828 29004 raft_consensus.cc:385] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.600996 29005 raft_consensus.cc:740] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.601230 29004 raft_consensus.cc:740] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.601583 29003 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.601851 29005 consensus_queue.cc:260] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.602362 29004 consensus_queue.cc:260] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.604993 29004 ts_tablet_manager.cc:1434] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:21:19.605985 29004 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.606384 29005 ts_tablet_manager.cc:1434] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.008s
I20260709 12:21:19.607443 29005 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.613322 29004 tablet_bootstrap.cc:654] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.613950 29005 tablet_bootstrap.cc:654] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.621678 29003 tablet_bootstrap.cc:654] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.621884 29004 tablet_bootstrap.cc:492] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.622469 29004 ts_tablet_manager.cc:1403] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 12:21:19.625633 29005 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.625557 29004 raft_consensus.cc:359] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.626358 29005 ts_tablet_manager.cc:1403] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.005s
I20260709 12:21:19.626387 29004 raft_consensus.cc:385] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.626895 29004 raft_consensus.cc:740] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.627777 29004 consensus_queue.cc:260] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.629912 29003 tablet_bootstrap.cc:492] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.630273 29004 ts_tablet_manager.cc:1434] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:19.630441 29003 ts_tablet_manager.cc:1403] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.008s
I20260709 12:21:19.630059 29005 raft_consensus.cc:359] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.630951 29005 raft_consensus.cc:385] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.631323 29005 raft_consensus.cc:740] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.631453 29004 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.632180 29005 consensus_queue.cc:260] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.633175 28926 raft_consensus.cc:493] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.633744 28926 raft_consensus.cc:515] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.634003 29003 raft_consensus.cc:359] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.634789 29003 raft_consensus.cc:385] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.635001 29005 ts_tablet_manager.cc:1434] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:21:19.635082 29003 raft_consensus.cc:740] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.636081 28926 leader_election.cc:290] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.636418 29005 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.636196 29003 consensus_queue.cc:260] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.638550 29003 ts_tablet_manager.cc:1434] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:21:19.639554 29003 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.643692 29005 tablet_bootstrap.cc:654] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.644503 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d033d6a3fe4eb19439c707068a7b10" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.645221 28859 raft_consensus.cc:2468] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.645985 29003 tablet_bootstrap.cc:654] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.647840 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d033d6a3fe4eb19439c707068a7b10" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.648895 28787 raft_consensus.cc:2468] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.651379 28676 leader_election.cc:304] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:19.658143 28926 raft_consensus.cc:2804] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.658666 28926 raft_consensus.cc:493] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.659125 28926 raft_consensus.cc:3060] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.666008 29005 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.666603 29005 ts_tablet_manager.cc:1403] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.011s
I20260709 12:21:19.666970 29003 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.667511 29003 ts_tablet_manager.cc:1403] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.011s
I20260709 12:21:19.670213 29005 raft_consensus.cc:359] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.670495 29003 raft_consensus.cc:359] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.671072 29005 raft_consensus.cc:385] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.671316 29003 raft_consensus.cc:385] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.671451 29005 raft_consensus.cc:740] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.671567 28992 raft_consensus.cc:493] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.671692 29003 raft_consensus.cc:740] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.672161 28992 raft_consensus.cc:515] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.672423 29005 consensus_queue.cc:260] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.673516 28946 raft_consensus.cc:493] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.673997 28946 raft_consensus.cc:515] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.674463 28992 leader_election.cc:290] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:19.675341 28926 raft_consensus.cc:515] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.676117 28946 leader_election.cc:290] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.676582 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:19.677218 28712 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:19.677842 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:19.678323 28926 leader_election.cc:290] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.678721 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.679451 28787 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
W20260709 12:21:19.680656 28750 leader_election.cc:343] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.681526 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.681830 29004 tablet_bootstrap.cc:654] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.682593 28824 leader_election.cc:304] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
W20260709 12:21:19.683094 28751 leader_election.cc:343] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.683663 28751 leader_election.cc:304] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616
I20260709 12:21:19.680982 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d033d6a3fe4eb19439c707068a7b10" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:19.685012 28859 raft_consensus.cc:3060] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.685477 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d033d6a3fe4eb19439c707068a7b10" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:19.686121 28787 raft_consensus.cc:3060] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.693857 28787 raft_consensus.cc:2468] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:19.694375 28946 raft_consensus.cc:2749] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:19.694834 28992 raft_consensus.cc:2804] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.701850 29004 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.706171 28992 raft_consensus.cc:493] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.706386 29004 ts_tablet_manager.cc:1403] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.013s
I20260709 12:21:19.706600 28992 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.707556 28678 leader_election.cc:304] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.708442 28926 raft_consensus.cc:2804] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.708913 28926 raft_consensus.cc:697] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:19.709529 29004 raft_consensus.cc:359] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.709718 28926 consensus_queue.cc:237] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.710342 29004 raft_consensus.cc:385] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.710649 29004 raft_consensus.cc:740] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.711392 29004 consensus_queue.cc:260] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.714896 29004 ts_tablet_manager.cc:1434] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:19.672629 29003 consensus_queue.cc:260] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.705384 29005 ts_tablet_manager.cc:1434] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.038s	user 0.019s	sys 0.019s
I20260709 12:21:19.716481 29005 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.716526 28992 raft_consensus.cc:515] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.718102 29004 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.703370 28859 raft_consensus.cc:2468] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:19.716997 29003 ts_tablet_manager.cc:1434] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 12:21:19.720563 29003 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.722401 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:19.723062 28711 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.723822 28992 leader_election.cc:290] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:19.725519 29004 tablet_bootstrap.cc:654] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.728782 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:19.729517 28787 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.740870 29005 tablet_bootstrap.cc:654] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.749203 29003 tablet_bootstrap.cc:654] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.756850 29004 tablet_bootstrap.cc:492] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.757349 28992 raft_consensus.cc:493] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.759737 28992 raft_consensus.cc:515] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.760524 29005 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.760988 29003 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.761057 29005 ts_tablet_manager.cc:1403] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.018s
I20260709 12:21:19.761664 29003 ts_tablet_manager.cc:1403] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.010s
I20260709 12:21:19.762601 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:19.762912 28786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
W20260709 12:21:19.764129 28824 leader_election.cc:343] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:19.764830 28827 leader_election.cc:343] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.765319 28827 leader_election.cc:304] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07; no voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858
I20260709 12:21:19.765659 28787 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:19.766126 28711 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:19.766367 29022 raft_consensus.cc:2749] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:19.757537 29004 ts_tablet_manager.cc:1403] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.022s
I20260709 12:21:19.766918 28827 leader_election.cc:304] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.764837 28609 catalog_manager.cc:5697] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.768464 28992 leader_election.cc:290] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:19.768494 29003 raft_consensus.cc:359] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.769344 28992 raft_consensus.cc:2804] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.769373 29003 raft_consensus.cc:385] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.770045 29003 raft_consensus.cc:740] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.770102 28992 raft_consensus.cc:697] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:19.770886 29003 consensus_queue.cc:260] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.771111 29004 raft_consensus.cc:359] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.771924 29004 raft_consensus.cc:385] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.772362 29004 raft_consensus.cc:740] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.771976 28992 consensus_queue.cc:237] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.773658 29003 ts_tablet_manager.cc:1434] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.006s
I20260709 12:21:19.774902 29003 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.775705 29005 raft_consensus.cc:359] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.777608 29005 raft_consensus.cc:385] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.778235 28926 raft_consensus.cc:493] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.779045 28926 raft_consensus.cc:515] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.778360 29005 raft_consensus.cc:740] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.782519 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4c0144eb8346fa8f1080616ea29109" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.773679 29004 consensus_queue.cc:260] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.782413 29005 consensus_queue.cc:260] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.784896 29004 ts_tablet_manager.cc:1434] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.006s
I20260709 12:21:19.785290 29005 ts_tablet_manager.cc:1434] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.004s
I20260709 12:21:19.785887 29004 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.786024 28859 raft_consensus.cc:2468] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.789247 28676 leader_election.cc:304] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:19.790297 29019 raft_consensus.cc:2804] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.790714 29019 raft_consensus.cc:493] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.791132 29019 raft_consensus.cc:3060] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.795499 29004 tablet_bootstrap.cc:654] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.796511 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4c0144eb8346fa8f1080616ea29109" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.797319 28787 raft_consensus.cc:2468] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.798599 28926 leader_election.cc:290] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.799638 29005 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.804147 28607 catalog_manager.cc:5697] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:19.806697 29019 raft_consensus.cc:515] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.806660 29005 tablet_bootstrap.cc:654] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.809754 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4c0144eb8346fa8f1080616ea29109" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:19.810343 28787 raft_consensus.cc:3060] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.822942 29003 tablet_bootstrap.cc:654] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.829388 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd4c0144eb8346fa8f1080616ea29109" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:19.830488 28859 raft_consensus.cc:3060] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.831393 29019 leader_election.cc:290] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.831765 28787 raft_consensus.cc:2468] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:19.833142 28678 leader_election.cc:304] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.835623 29019 raft_consensus.cc:2804] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.835721 29004 tablet_bootstrap.cc:492] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.835726 29003 tablet_bootstrap.cc:492] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.836495 29004 ts_tablet_manager.cc:1403] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.015s
I20260709 12:21:19.835961 29005 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.836815 29003 ts_tablet_manager.cc:1403] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.005s
I20260709 12:21:19.837271 29005 ts_tablet_manager.cc:1403] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.010s
I20260709 12:21:19.836253 29019 raft_consensus.cc:697] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:19.838510 28859 raft_consensus.cc:2468] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:19.839634 29019 consensus_queue.cc:237] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.839964 29003 raft_consensus.cc:359] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.840711 29003 raft_consensus.cc:385] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.840749 29005 raft_consensus.cc:359] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.841171 29003 raft_consensus.cc:740] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.841580 29005 raft_consensus.cc:385] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.841903 29005 raft_consensus.cc:740] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.842144 29003 consensus_queue.cc:260] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.842916 29005 consensus_queue.cc:260] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.844743 29003 ts_tablet_manager.cc:1434] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:19.841585 29004 raft_consensus.cc:359] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.845297 29005 ts_tablet_manager.cc:1434] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 12:21:19.845618 29004 raft_consensus.cc:385] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.845831 29003 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.845970 29004 raft_consensus.cc:740] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.846390 29005 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.846777 29004 consensus_queue.cc:260] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.852994 29003 tablet_bootstrap.cc:654] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.853853 29005 tablet_bootstrap.cc:654] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.856372 29004 ts_tablet_manager.cc:1434] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:21:19.857481 29004 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.866868 29004 tablet_bootstrap.cc:654] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.868074 29003 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.868743 29003 ts_tablet_manager.cc:1403] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.006s
I20260709 12:21:19.878186 29003 raft_consensus.cc:359] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.878890 29003 raft_consensus.cc:385] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.879221 29003 raft_consensus.cc:740] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.879525 29005 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.879696 29004 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.880059 29005 ts_tablet_manager.cc:1403] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.034s	user 0.028s	sys 0.004s
I20260709 12:21:19.880512 29004 ts_tablet_manager.cc:1403] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.006s
I20260709 12:21:19.879944 29003 consensus_queue.cc:260] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.881726 28610 catalog_manager.cc:5697] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.883066 29005 raft_consensus.cc:359] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.883582 29003 ts_tablet_manager.cc:1434] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:21:19.883806 29005 raft_consensus.cc:385] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.883575 29004 raft_consensus.cc:359] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.884389 29005 raft_consensus.cc:740] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.884744 29004 raft_consensus.cc:385] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.884985 29003 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: Bootstrap starting.
I20260709 12:21:19.885053 29004 raft_consensus.cc:740] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.885320 29005 consensus_queue.cc:260] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.887523 29005 ts_tablet_manager.cc:1434] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:19.888453 29005 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: Bootstrap starting.
I20260709 12:21:19.892478 29020 raft_consensus.cc:493] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.893276 29003 tablet_bootstrap.cc:654] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.893002 29020 raft_consensus.cc:515] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.895193 29020 leader_election.cc:290] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.897152 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1531185db849d8bcb282065ffaa5c6" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.901006 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1531185db849d8bcb282065ffaa5c6" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
W20260709 12:21:19.902889 28750 leader_election.cc:343] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.908344 29003 tablet_bootstrap.cc:492] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: No bootstrap required, opened a new log
I20260709 12:21:19.908942 29003 ts_tablet_manager.cc:1403] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.011s
I20260709 12:21:19.910216 29019 raft_consensus.cc:493] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.910708 29019 raft_consensus.cc:515] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.886075 29004 consensus_queue.cc:260] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.911815 29003 raft_consensus.cc:359] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.912297 28859 raft_consensus.cc:2468] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:19.912523 29003 raft_consensus.cc:385] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.912880 29003 raft_consensus.cc:740] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.913499 29003 consensus_queue.cc:260] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.914034 29004 ts_tablet_manager.cc:1434] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:21:19.914217 28751 leader_election.cc:304] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 931f2597e137468ea1700bfa37832616
I20260709 12:21:19.915004 29020 raft_consensus.cc:2804] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.915380 29020 raft_consensus.cc:493] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.915709 29020 raft_consensus.cc:3060] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.916036 29004 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:19.922531 29004 tablet_bootstrap.cc:654] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:19.915657 29003 ts_tablet_manager.cc:1434] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:19.923794 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.924175 29020 raft_consensus.cc:515] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.909929 29005 tablet_bootstrap.cc:654] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:19.926364 28678 leader_election.cc:343] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.926460 29020 leader_election.cc:290] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.927726 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.928798 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1531185db849d8bcb282065ffaa5c6" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:19.929437 28712 raft_consensus.cc:3060] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.930889 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1531185db849d8bcb282065ffaa5c6" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:19.931694 28860 raft_consensus.cc:3060] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.934947 28712 raft_consensus.cc:2468] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:19.935910 28926 raft_consensus.cc:493] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:19.936756 28676 leader_election.cc:343] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.912881 29019 leader_election.cc:290] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.937482 28926 raft_consensus.cc:515] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.937494 28676 leader_election.cc:304] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 931f2597e137468ea1700bfa37832616; no voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858
I20260709 12:21:19.938838 29019 raft_consensus.cc:2749] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:19.939673 28926 leader_election.cc:290] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.940670 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.941500 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.942094 28787 raft_consensus.cc:2468] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.943223 28678 leader_election.cc:304] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.947211 28926 raft_consensus.cc:493] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.947675 28926 raft_consensus.cc:515] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.949954 28926 leader_election.cc:290] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.950598 28926 raft_consensus.cc:2804] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.950955 28926 raft_consensus.cc:493] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.949795 28860 raft_consensus.cc:2468] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:19.951346 28926 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.954101 29005 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: No bootstrap required, opened a new log
I20260709 12:21:19.954773 29005 ts_tablet_manager.cc:1403] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.006s
I20260709 12:21:19.954982 28750 leader_election.cc:304] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.956099 29039 raft_consensus.cc:493] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.956676 28946 raft_consensus.cc:2804] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.956528 29039 raft_consensus.cc:515] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.958452 29005 raft_consensus.cc:359] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.958675 29039 leader_election.cc:290] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
W20260709 12:21:19.958756 28676 leader_election.cc:343] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.959280 29005 raft_consensus.cc:385] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.959672 29005 raft_consensus.cc:740] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.960314 29005 consensus_queue.cc:260] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.962992 29005 ts_tablet_manager.cc:1434] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:21:19.963507 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.964413 28859 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:19.963027 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:19.957212 28926 raft_consensus.cc:515] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.965763 28751 leader_election.cc:304] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.966967 29039 raft_consensus.cc:2804] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.967388 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:19.967396 29039 raft_consensus.cc:493] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.967506 28860 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.968120 28787 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:19.968472 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:19.969328 29039 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.969487 28712 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 0.
I20260709 12:21:19.969475 28676 leader_election.cc:304] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:19.974469 29004 tablet_bootstrap.cc:492] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:19.974591 28926 leader_election.cc:290] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.975008 29004 ts_tablet_manager.cc:1403] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.017s
I20260709 12:21:19.975342 28926 raft_consensus.cc:2804] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.975615 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:19.975653 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:19.976315 28787 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:19.977496 28676 leader_election.cc:343] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:19.977680 29004 raft_consensus.cc:359] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.978340 29004 raft_consensus.cc:385] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:19.978565 29004 raft_consensus.cc:740] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:19.979149 29004 consensus_queue.cc:260] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.979347 28946 raft_consensus.cc:697] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:19.980319 28946 consensus_queue.cc:237] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.975790 28926 raft_consensus.cc:493] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.983057 29039 raft_consensus.cc:515] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.984231 28926 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.986115 29039 leader_election.cc:290] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:19.986853 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:19.987452 28712 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.987776 28787 raft_consensus.cc:2468] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:19.987869 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:19.988516 28860 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.989084 28678 leader_election.cc:304] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:19.989822 28992 raft_consensus.cc:493] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:19.990275 29019 raft_consensus.cc:2804] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.990340 28992 raft_consensus.cc:515] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.992619 28992 leader_election.cc:290] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:19.994216 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:19.994876 28711 raft_consensus.cc:2468] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:19.994930 28712 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:19.996026 28860 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 1.
I20260709 12:21:19.996197 28824 leader_election.cc:304] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:19.996390 28750 leader_election.cc:304] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:19.997455 29039 raft_consensus.cc:2804] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:19.997916 29039 raft_consensus.cc:697] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:19.998255 28992 raft_consensus.cc:2804] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:19.998620 28992 raft_consensus.cc:493] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:19.998917 28992 raft_consensus.cc:3060] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:19.999328 29004 ts_tablet_manager.cc:1434] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:21:19.998816 29039 consensus_queue.cc:237] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:19.998723 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:20.000214 28787 raft_consensus.cc:2468] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 0.
I20260709 12:21:20.000891 29004 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: Bootstrap starting.
I20260709 12:21:20.006619 28992 raft_consensus.cc:515] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.007534 29019 raft_consensus.cc:697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.008759 29019 consensus_queue.cc:237] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.010496 29004 tablet_bootstrap.cc:654] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:20.013414 28926 raft_consensus.cc:515] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.016513 28926 leader_election.cc:290] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.016067 28607 catalog_manager.cc:5697] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.018129 28786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:20.018951 28786 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.016621 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.023600 28860 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.037436 28610 catalog_manager.cc:5697] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 1 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.052577 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:20.053602 28712 raft_consensus.cc:3060] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.063087 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489cf4ccbbeb4458ab1d3ca94f1f89ed" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:20.063478 28992 leader_election.cc:290] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:20.063827 28787 raft_consensus.cc:3060] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.064745 28712 raft_consensus.cc:2468] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:20.065053 28786 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:20.066263 28860 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:20.066825 28824 leader_election.cc:304] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:20.067584 28676 leader_election.cc:304] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:20.068589 28926 raft_consensus.cc:2804] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:20.069029 28992 raft_consensus.cc:2804] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:20.069119 28926 raft_consensus.cc:697] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.070283 28926 consensus_queue.cc:237] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.070917 28992 raft_consensus.cc:697] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:20.071890 29004 tablet_bootstrap.cc:492] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: No bootstrap required, opened a new log
I20260709 12:21:20.071803 28992 consensus_queue.cc:237] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.072444 29004 ts_tablet_manager.cc:1403] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: Time spent bootstrapping tablet: real 0.072s	user 0.036s	sys 0.026s
I20260709 12:21:20.075455 29004 raft_consensus.cc:359] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.076082 29004 raft_consensus.cc:385] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:20.076402 29004 raft_consensus.cc:740] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.077025 29004 consensus_queue.cc:260] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.076743 28610 catalog_manager.cc:5697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.077911 28787 raft_consensus.cc:2468] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 1.
I20260709 12:21:20.079393 29004 ts_tablet_manager.cc:1434] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:20.086455 28609 catalog_manager.cc:5697] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:20.090278 28992 raft_consensus.cc:493] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:20.090857 28992 raft_consensus.cc:515] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.091943 28610 catalog_manager.cc:5697] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.093082 28992 leader_election.cc:290] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:20.094192 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:20.094579 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931f2597e137468ea1700bfa37832616" is_pre_election: true
I20260709 12:21:20.094923 28787 raft_consensus.cc:2393] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8538ce0a884c4a10ae31c81b33103c07 in current term 1: Already voted for candidate 931f2597e137468ea1700bfa37832616 in this term.
I20260709 12:21:20.097306 28824 leader_election.cc:304] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07; no voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858
I20260709 12:21:20.098101 28992 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.105716 28992 raft_consensus.cc:2749] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:20.170805 28992 consensus_queue.cc:1048] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.175381 28926 raft_consensus.cc:493] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:20.175899 28926 raft_consensus.cc:515] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.178726 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858" is_pre_election: true
I20260709 12:21:20.179647 28926 leader_election.cc:290] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.180292 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07" is_pre_election: true
I20260709 12:21:20.179337 28787 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:20.180922 28860 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 0.
I20260709 12:21:20.182199 28678 leader_election.cc:304] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:20.184118 29019 raft_consensus.cc:2804] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:20.184016 28992 consensus_queue.cc:1048] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.184653 29019 raft_consensus.cc:493] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:20.185113 29019 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.192860 29019 raft_consensus.cc:515] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.195955 28786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:20.196571 28786 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.197359 29019 leader_election.cc:290] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.197578 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.198211 28860 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:20.198834 29019 consensus_queue.cc:1048] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.204118 28786 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:20.205322 28678 leader_election.cc:304] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:20.206494 29019 raft_consensus.cc:2804] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:20.207039 29019 raft_consensus.cc:697] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.207937 29019 consensus_queue.cc:237] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.205294 28860 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 1.
I20260709 12:21:20.215916 28926 consensus_queue.cc:1048] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.221215 28610 catalog_manager.cc:5697] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 1 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.386363 28926 consensus_queue.cc:1048] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.396938 29019 consensus_queue.cc:1048] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.416816 28946 consensus_queue.cc:1048] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.428169 29020 consensus_queue.cc:1048] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.443750 28859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
mode: GRACEFUL
new_leader_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:39246
I20260709 12:21:20.444257 28859 raft_consensus.cc:606] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Received request to transfer leadership to TS 931f2597e137468ea1700bfa37832616
I20260709 12:21:20.448886 28787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:52424
I20260709 12:21:20.449370 28787 raft_consensus.cc:606] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Received request to transfer leadership to TS 931f2597e137468ea1700bfa37832616
I20260709 12:21:20.466153 28712 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60"
dest_uuid: "931f2597e137468ea1700bfa37832616"
mode: GRACEFUL
new_leader_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:42344
I20260709 12:21:20.466719 28712 raft_consensus.cc:606] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Received request to transfer leadership to TS d1945ee0314e465a875344905594c858
I20260709 12:21:20.467976 29042 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.474308 28992 consensus_queue.cc:1048] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.475313 28711 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7"
dest_uuid: "931f2597e137468ea1700bfa37832616"
mode: GRACEFUL
new_leader_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:42344
I20260709 12:21:20.475944 28711 raft_consensus.cc:606] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Received request to transfer leadership to TS d1945ee0314e465a875344905594c858
I20260709 12:21:20.476069 29042 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.480296 29039 raft_consensus.cc:993] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: : Instructing follower 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:20.480811 29039 raft_consensus.cc:1081] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Signalling peer 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:20.483687 28711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347"
dest_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:42328
I20260709 12:21:20.484301 28711 raft_consensus.cc:493] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.484750 28711 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.487180 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4"
dest_uuid: "931f2597e137468ea1700bfa37832616"
mode: GRACEFUL
new_leader_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:42344
I20260709 12:21:20.487917 28710 raft_consensus.cc:606] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Received request to transfer leadership to TS d1945ee0314e465a875344905594c858
I20260709 12:21:20.493337 28711 raft_consensus.cc:515] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.496606 28992 consensus_queue.cc:1048] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.504772 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.507992 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:20.508687 28787 raft_consensus.cc:3055] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.509042 28787 raft_consensus.cc:740] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:20.512480 28711 leader_election.cc:290] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.519083 28787 consensus_queue.cc:260] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.520421 28787 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.538028 28787 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:20.540248 28678 leader_election.cc:304] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:20.541997 28926 raft_consensus.cc:2804] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.542610 28926 raft_consensus.cc:697] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.544102 28926 consensus_queue.cc:237] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.553194 28926 consensus_queue.cc:1048] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.555804 28607 catalog_manager.cc:5697] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 1 to 2, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.565501 29046 consensus_queue.cc:1048] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:20.580274 29019 raft_consensus.cc:993] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: : Instructing follower d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.580762 29019 raft_consensus.cc:1081] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Signalling peer d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.582324 28787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7"
dest_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:52416
I20260709 12:21:20.582831 28787 raft_consensus.cc:493] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.583199 28787 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.588284 28787 raft_consensus.cc:515] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.590957 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:20.591305 28787 leader_election.cc:290] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.591480 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb77d6947fd44daf919b6e8af52861e7" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.591537 28712 raft_consensus.cc:3055] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.592150 28859 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.592332 28712 raft_consensus.cc:740] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.593098 28712 consensus_queue.cc:260] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.594295 28712 raft_consensus.cc:3060] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.599026 28859 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:20.601161 28712 raft_consensus.cc:2468] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:20.602304 28751 leader_election.cc:304] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:20.603332 29019 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:20.605113 29039 raft_consensus.cc:2804] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.605782 29039 raft_consensus.cc:697] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:20.606704 29039 consensus_queue.cc:237] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.611346 29046 raft_consensus.cc:993] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: : Instructing follower d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.611845 28926 raft_consensus.cc:1081] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Signalling peer d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.615883 28787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4"
dest_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:52416
I20260709 12:21:20.616510 28787 raft_consensus.cc:493] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.616865 28787 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.619168 29019 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:20.621959 28609 catalog_manager.cc:5697] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 1 to 2, leader changed from 931f2597e137468ea1700bfa37832616 (127.16.193.65) to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.624843 28787 raft_consensus.cc:515] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.627086 28787 leader_election.cc:290] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.627743 28712 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:20.628324 28712 raft_consensus.cc:3055] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.628594 28712 raft_consensus.cc:740] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.629285 28712 consensus_queue.cc:260] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.629648 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.630218 28712 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.631628 29067 raft_consensus.cc:993] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: : Instructing follower 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:20.632093 28992 raft_consensus.cc:1081] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Signalling peer 931f2597e137468ea1700bfa37832616 to start an election
I20260709 12:21:20.634532 28711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89"
dest_uuid: "931f2597e137468ea1700bfa37832616"
 from {username='slave'} at 127.0.0.1:42330
I20260709 12:21:20.635123 28711 raft_consensus.cc:493] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.635558 28711 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.641059 28712 raft_consensus.cc:2468] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:20.645972 28711 raft_consensus.cc:515] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.647940 28711 leader_election.cc:290] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Requested vote from peers d1945ee0314e465a875344905594c858 (127.16.193.66:45499), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.648634 28750 leader_election.cc:304] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:20.649607 29042 raft_consensus.cc:2804] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.650115 29042 raft_consensus.cc:697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:20.650236 29019 consensus_queue.cc:1048] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:20.650800 28860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.650949 29042 consensus_queue.cc:237] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.651966 28787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2889961de7345ee9e307539e41a5a89" candidate_uuid: "931f2597e137468ea1700bfa37832616" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:20.652551 28787 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.651453 28860 raft_consensus.cc:3055] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.655323 28860 raft_consensus.cc:740] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:20.656127 28860 consensus_queue.cc:260] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.657524 28860 raft_consensus.cc:3060] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.658978 28787 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:20.660670 28678 leader_election.cc:304] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931f2597e137468ea1700bfa37832616, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:20.661509 29019 raft_consensus.cc:2804] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.662098 29019 raft_consensus.cc:697] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Becoming Leader. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.661074 28610 catalog_manager.cc:5697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 1 to 2, leader changed from 931f2597e137468ea1700bfa37832616 (127.16.193.65) to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.662990 29019 consensus_queue.cc:237] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.665117 28860 raft_consensus.cc:2468] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931f2597e137468ea1700bfa37832616 in term 2.
I20260709 12:21:20.671753 28607 catalog_manager.cc:5697] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 reported cstate change: term changed from 1 to 2, leader changed from 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67) to 931f2597e137468ea1700bfa37832616 (127.16.193.65). New cstate: current_term: 2 leader_uuid: "931f2597e137468ea1700bfa37832616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.677876 29019 consensus_queue.cc:1048] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:20.686304 29044 raft_consensus.cc:993] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: : Instructing follower d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.686746 29044 raft_consensus.cc:1081] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Signalling peer d1945ee0314e465a875344905594c858 to start an election
I20260709 12:21:20.689023 28787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60"
dest_uuid: "d1945ee0314e465a875344905594c858"
 from {username='slave'} at 127.0.0.1:52416
I20260709 12:21:20.689591 28787 raft_consensus.cc:493] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.689939 28787 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.694988 28787 raft_consensus.cc:515] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.696483 28787 leader_election.cc:290] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67:45905)
I20260709 12:21:20.697649 28859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
I20260709 12:21:20.697762 28711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b575713ccdc48bfb6616b5e1afc0c60" candidate_uuid: "d1945ee0314e465a875344905594c858" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:20.698277 28859 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.698451 28711 raft_consensus.cc:3055] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.698761 28711 raft_consensus.cc:740] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:20.699541 28711 consensus_queue.cc:260] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.700963 28711 raft_consensus.cc:3060] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.703766 28859 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:20.704917 28751 leader_election.cc:304] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:20.705673 29042 raft_consensus.cc:2804] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.706223 29042 raft_consensus.cc:697] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Leader. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:20.707104 29042 consensus_queue.cc:237] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:20.707839 28711 raft_consensus.cc:2468] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1945ee0314e465a875344905594c858 in term 2.
I20260709 12:21:20.715581 28609 catalog_manager.cc:5697] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 reported cstate change: term changed from 1 to 2, leader changed from 931f2597e137468ea1700bfa37832616 (127.16.193.65) to d1945ee0314e465a875344905594c858 (127.16.193.66). New cstate: current_term: 2 leader_uuid: "d1945ee0314e465a875344905594c858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.058009 28859 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:21.065466 28859 raft_consensus.cc:1275] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.066743 28787 raft_consensus.cc:1275] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.066807 29039 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:21.068094 29046 consensus_queue.cc:1048] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.076980 28711 raft_consensus.cc:1275] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.082258 28859 raft_consensus.cc:1275] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.080156 29039 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:21:21.088629 28926 consensus_queue.cc:1048] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.107013 28859 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:21.111563 28859 raft_consensus.cc:1275] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.112728 28926 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.119881 28787 raft_consensus.cc:1275] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 931f2597e137468ea1700bfa37832616: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.122032 29046 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.186470 28859 raft_consensus.cc:1275] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.187755 29039 consensus_queue.cc:1048] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.198382 28711 raft_consensus.cc:1275] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.199977 29039 consensus_queue.cc:1048] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.204618 28860 raft_consensus.cc:1275] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.206049 29039 consensus_queue.cc:1048] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.213851 28711 raft_consensus.cc:1275] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer d1945ee0314e465a875344905594c858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.216104 29076 consensus_queue.cc:1048] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:23.514938 28607 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57540:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:21:23.554909 28607 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57540:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:21:23.601521 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:21:23.815940 29103 log.cc:927] Time spent T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:21:23.870092 29118 log.cc:927] Time spent T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Append to log took a long time: real 0.058s	user 0.003s	sys 0.000s
W20260709 12:21:23.871801 29115 log.cc:927] Time spent T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Append to log took a long time: real 0.073s	user 0.004s	sys 0.000s
W20260709 12:21:23.921471 29120 log.cc:927] Time spent T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:21:23.973047 29122 log.cc:927] Time spent T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
W20260709 12:21:23.973555 29123 log.cc:927] Time spent T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Append to log took a long time: real 0.162s	user 0.000s	sys 0.003s
I20260709 12:21:23.992251 28708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347"
dest_uuid: "931f2597e137468ea1700bfa37832616"
mode: GRACEFUL
new_leader_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:60794
I20260709 12:21:23.993340 28708 raft_consensus.cc:606] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Received request to transfer leadership to TS 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:23.993395 29131 tablet.cc:1724] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:23.996011 29125 log.cc:927] Time spent T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:21:23.995502 29129 tablet.cc:1724] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:23.998731 29133 tablet.cc:1724] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:23.995922 29136 tablet.cc:1724] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.015024 29132 tablet.cc:1724] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.023857 29141 tablet.cc:1724] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.024425 29144 tablet.cc:1724] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:24.027267 28887 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:5d0fefc8bf46465ea8fd628770cd4ba4)
W20260709 12:21:24.029789 28887 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:488a5535ca71476db152cd1b9dfe5347)
I20260709 12:21:24.086305 29142 tablet.cc:1724] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:24.099611 28738 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:02d033d6a3fe4eb19439c707068a7b10)
I20260709 12:21:24.024546 29143 tablet.cc:1724] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.150456 29150 tablet.cc:1724] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.148228 29148 tablet.cc:1724] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.154516 29132 tablet.cc:1724] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.153239 29129 tablet.cc:1724] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.153175 29146 tablet.cc:1724] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.147017 29136 tablet.cc:1724] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.158582 29131 tablet.cc:1724] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.163450 29133 tablet.cc:1724] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.174645 29142 tablet.cc:1724] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.181232 29131 tablet.cc:1724] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.187367 29141 tablet.cc:1724] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:24.207201 28676 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:45905, user_credentials={real_user=slave}} blocked reactor thread for 64836.7us
I20260709 12:21:24.194849 29143 tablet.cc:1724] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.209901 28783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:60980
I20260709 12:21:24.210579 28783 raft_consensus.cc:606] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Received request to transfer leadership to TS 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:24.212344 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:21:24.212946 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:24.221423 28783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f"
dest_uuid: "d1945ee0314e465a875344905594c858"
mode: GRACEFUL
new_leader_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:60980
I20260709 12:21:24.221918 28783 raft_consensus.cc:606] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Received request to transfer leadership to TS 8538ce0a884c4a10ae31c81b33103c07
W20260709 12:21:24.223987 28678 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:45499, user_credentials={real_user=slave}} blocked reactor thread for 64669.6us
I20260709 12:21:24.227409 29141 tablet.cc:1724] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.229147 29144 tablet.cc:1724] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.232738 28710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9"
dest_uuid: "931f2597e137468ea1700bfa37832616"
mode: GRACEFUL
new_leader_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:60794
I20260709 12:21:24.233373 28710 raft_consensus.cc:606] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Received request to transfer leadership to TS 8538ce0a884c4a10ae31c81b33103c07
I20260709 12:21:24.235415 29129 tablet.cc:1724] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.238171 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:24.238833 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:24.239871 29136 tablet.cc:1724] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.246335 29148 tablet.cc:1724] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.263382 29113 raft_consensus.cc:993] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: : Instructing follower 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.264108 29076 raft_consensus.cc:1081] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Signalling peer 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.229849 29133 tablet.cc:1724] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:24.267092 28860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:39236
I20260709 12:21:24.267767 28860 raft_consensus.cc:493] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.268190 28860 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.277172 28860 raft_consensus.cc:515] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.280227 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:24.281500 28608 catalog_manager.cc:6242] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: soft_deleted_table [id=3563ae5834b242e69d97fcfd2d3c3c8e] alter complete (version 1)
I20260709 12:21:24.281080 28709 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.282143 28783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fefc8bf46465ea8fd628770cd4ba4" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:24.282840 28783 raft_consensus.cc:3055] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:24.283156 28783 raft_consensus.cc:740] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:24.284063 28783 consensus_queue.cc:260] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.285295 28783 raft_consensus.cc:3060] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.293397 28783 raft_consensus.cc:2468] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 3.
I20260709 12:21:24.293869 28709 raft_consensus.cc:2468] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 3.
I20260709 12:21:24.281038 28860 leader_election.cc:290] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 3 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:24.294756 28827 leader_election.cc:304] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:24.300431 29154 raft_consensus.cc:2804] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:24.300938 29154 raft_consensus.cc:697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:24.301950 29154 consensus_queue.cc:237] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.311118 28610 catalog_manager.cc:5697] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 2 to 3, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.424934 29152 raft_consensus.cc:993] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: : Instructing follower 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.425562 29153 raft_consensus.cc:1081] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Signalling peer 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.427109 28855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:39238
I20260709 12:21:24.427716 28855 raft_consensus.cc:493] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.428026 28855 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.433982 28855 raft_consensus.cc:515] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.435779 28855 leader_election.cc:290] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 3 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:24.436457 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:24.436826 28783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488a5535ca71476db152cd1b9dfe5347" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:24.437072 28709 raft_consensus.cc:3055] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:24.437445 28783 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.437589 28709 raft_consensus.cc:740] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:24.438443 28709 consensus_queue.cc:260] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.439503 28709 raft_consensus.cc:3060] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.443383 28783 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 3.
I20260709 12:21:24.444638 28827 leader_election.cc:304] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:24.445509 29154 raft_consensus.cc:2804] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:24.445968 29154 raft_consensus.cc:697] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:24.446890 29154 consensus_queue.cc:237] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.447206 28709 raft_consensus.cc:2468] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 3.
I20260709 12:21:24.453676 28610 catalog_manager.cc:5697] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 2 to 3, leader changed from 931f2597e137468ea1700bfa37832616 (127.16.193.65) to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.479604 29152 raft_consensus.cc:993] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: : Instructing follower 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.480091 29152 raft_consensus.cc:1081] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Signalling peer 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.481793 28855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:39238
I20260709 12:21:24.482316 28855 raft_consensus.cc:493] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.482625 28855 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.488240 28855 raft_consensus.cc:515] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.490343 28855 leader_election.cc:290] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:24.490921 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:24.491405 28783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88c811c463e64eff91a0ed703c04cce9" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:24.491662 28709 raft_consensus.cc:3055] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:24.492061 28783 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.492077 28709 raft_consensus.cc:740] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 931f2597e137468ea1700bfa37832616, State: Running, Role: LEADER
I20260709 12:21:24.493018 28709 consensus_queue.cc:260] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.494071 28709 raft_consensus.cc:3060] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.497643 28783 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 2.
I20260709 12:21:24.498821 28827 leader_election.cc:304] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, d1945ee0314e465a875344905594c858; no voters: 
I20260709 12:21:24.499645 29154 raft_consensus.cc:2804] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:24.499797 28709 raft_consensus.cc:2468] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 2.
I20260709 12:21:24.500177 29154 raft_consensus.cc:697] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:24.501226 29154 consensus_queue.cc:237] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.509352 28610 catalog_manager.cc:5697] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 1 to 2, leader changed from 931f2597e137468ea1700bfa37832616 (127.16.193.65) to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.691771 28783 raft_consensus.cc:1275] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:24.693033 29154 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.701813 28709 raft_consensus.cc:1275] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:24.703297 28992 consensus_queue.cc:1048] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:24.715519 29113 raft_consensus.cc:993] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: : Instructing follower 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.715978 29076 raft_consensus.cc:1081] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Signalling peer 8538ce0a884c4a10ae31c81b33103c07 to start an election
I20260709 12:21:24.717866 28860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f"
dest_uuid: "8538ce0a884c4a10ae31c81b33103c07"
 from {username='slave'} at 127.0.0.1:39236
I20260709 12:21:24.718400 28860 raft_consensus.cc:493] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.718747 28860 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.725561 28860 raft_consensus.cc:515] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.727048 28860 leader_election.cc:290] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Requested vote from peers 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699), d1945ee0314e465a875344905594c858 (127.16.193.66:45499)
I20260709 12:21:24.727936 28709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "931f2597e137468ea1700bfa37832616"
I20260709 12:21:24.728260 28783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22170366e5214d469b9f37d4c7c7c30f" candidate_uuid: "8538ce0a884c4a10ae31c81b33103c07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1945ee0314e465a875344905594c858"
I20260709 12:21:24.728730 28709 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.728906 28783 raft_consensus.cc:3055] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:24.729347 28783 raft_consensus.cc:740] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1945ee0314e465a875344905594c858, State: Running, Role: LEADER
I20260709 12:21:24.730247 28783 consensus_queue.cc:260] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.731299 28783 raft_consensus.cc:3060] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.734321 28709 raft_consensus.cc:2468] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 2.
I20260709 12:21:24.735600 28824 leader_election.cc:304] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8538ce0a884c4a10ae31c81b33103c07, 931f2597e137468ea1700bfa37832616; no voters: 
I20260709 12:21:24.736366 28992 raft_consensus.cc:2804] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:24.736769 28992 raft_consensus.cc:697] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Becoming Leader. State: Replica: 8538ce0a884c4a10ae31c81b33103c07, State: Running, Role: LEADER
I20260709 12:21:24.736963 28783 raft_consensus.cc:2468] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8538ce0a884c4a10ae31c81b33103c07 in term 2.
I20260709 12:21:24.737655 28992 consensus_queue.cc:237] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } }
I20260709 12:21:24.744895 28607 catalog_manager.cc:5697] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 reported cstate change: term changed from 1 to 2, leader changed from d1945ee0314e465a875344905594c858 (127.16.193.66) to 8538ce0a884c4a10ae31c81b33103c07 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "8538ce0a884c4a10ae31c81b33103c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8538ce0a884c4a10ae31c81b33103c07" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 45905 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.851091 28783 raft_consensus.cc:1275] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:24.852451 28992 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.861140 28709 raft_consensus.cc:1275] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:24.862500 29154 consensus_queue.cc:1048] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.911382 28783 raft_consensus.cc:1275] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:24.912542 29154 consensus_queue.cc:1048] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:24.922209 28709 raft_consensus.cc:1275] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:24.923599 28992 consensus_queue.cc:1048] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:25.217789 28783 raft_consensus.cc:1275] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:25.219194 28992 consensus_queue.cc:1048] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1945ee0314e465a875344905594c858" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:25.229373 28709 raft_consensus.cc:1275] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Refusing update from remote peer 8538ce0a884c4a10ae31c81b33103c07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:25.230767 28992 consensus_queue.cc:1048] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931f2597e137468ea1700bfa37832616" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:25.239997 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:25.246672 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:25.247192 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:25.254674 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:25.255273 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:26.350997 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:26.375106 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:26.376282 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:26.385538 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:26.386121 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:21:27.382189 28808 debug-util.cc:398] Leaking SignalData structure 0x7b080007f8a0 after lost signal to thread 17160
W20260709 12:21:27.383306 28808 debug-util.cc:398] Leaking SignalData structure 0x7b08000320e0 after lost signal to thread 28644
W20260709 12:21:27.384392 28808 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8f60 after lost signal to thread 28736
W20260709 12:21:27.385416 28808 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf4a0 after lost signal to thread 28811
W20260709 12:21:27.386466 28808 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9740 after lost signal to thread 28885
I20260709 12:21:27.390506 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:27.399320 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:27.399886 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:27.407887 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:27.408434 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:28.409453 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:28.414770 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:28.415315 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:28.420239 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:28.420619 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:29.421785 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:29.430475 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:29.431048 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:29.438987 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:29.439529 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:30.440665 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:30.445726 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:30.446121 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:30.452829 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:30.453301 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:31.454419 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:31.461515 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:31.462230 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:31.468092 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:31.468660 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:32.469838 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:32.474833 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:32.475294 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:32.481134 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:32.481890 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:33.483637 17157 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:33.492000 17157 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:33.492430 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:33.498274 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:33.498660 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:34.558903 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
W20260709 12:21:34.563907 28750 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:21:34.569073 28750 consensus_peers.cc:597] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:34.579378 28750 consensus_peers.cc:597] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:34.607914 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:34.609056 17157 tablet_replica.cc:333] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.610610 17157 raft_consensus.cc:2243] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.611791 17157 raft_consensus.cc:2272] T 1e1531185db849d8bcb282065ffaa5c6 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.615707 17157 tablet_replica.cc:333] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.616652 17157 raft_consensus.cc:2243] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.617491 17157 raft_consensus.cc:2272] T 1b575713ccdc48bfb6616b5e1afc0c60 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.620692 17157 tablet_replica.cc:333] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.621398 17157 raft_consensus.cc:2243] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.621970 17157 raft_consensus.cc:2272] T fb77d6947fd44daf919b6e8af52861e7 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.624406 17157 tablet_replica.cc:333] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.625144 17157 raft_consensus.cc:2243] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.626041 17157 raft_consensus.cc:2272] T dd4c0144eb8346fa8f1080616ea29109 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.629241 17157 tablet_replica.cc:333] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.630097 17157 raft_consensus.cc:2243] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.630648 17157 raft_consensus.cc:2272] T 488a5535ca71476db152cd1b9dfe5347 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.633205 17157 tablet_replica.cc:333] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.633737 17157 raft_consensus.cc:2243] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.634611 17157 raft_consensus.cc:2272] T 02d033d6a3fe4eb19439c707068a7b10 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.637195 17157 tablet_replica.cc:333] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.637837 17157 raft_consensus.cc:2243] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.638396 17157 raft_consensus.cc:2272] T 17e3fa379b7d4e20ad78550c233c2389 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.640812 17157 tablet_replica.cc:333] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.641405 17157 raft_consensus.cc:2243] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.641984 17157 raft_consensus.cc:2272] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.644335 17157 tablet_replica.cc:333] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.644953 17157 raft_consensus.cc:2243] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.645437 17157 raft_consensus.cc:2272] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 931f2597e137468ea1700bfa37832616 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.647898 17157 tablet_replica.cc:333] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.648612 17157 raft_consensus.cc:2243] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.649219 17157 raft_consensus.cc:2272] T 58fdc9b9c1cc431897d0f1242d1fdccf P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.651971 17157 tablet_replica.cc:333] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.652649 17157 raft_consensus.cc:2243] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.653208 17157 raft_consensus.cc:2272] T 091ceec6e2ad4b22be7737dcaf361b63 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.655972 17157 tablet_replica.cc:333] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.656567 17157 raft_consensus.cc:2243] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.657107 17157 raft_consensus.cc:2272] T 3ce98f1bed204d93ad21d02af807727c P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.659660 17157 tablet_replica.cc:333] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.660204 17157 raft_consensus.cc:2243] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.661541 17157 raft_consensus.cc:2272] T 0e543962a89c490a9572dc8a6ef7c792 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.664976 17157 tablet_replica.cc:333] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.665637 17157 raft_consensus.cc:2243] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.666332 17157 raft_consensus.cc:2272] T 88c811c463e64eff91a0ed703c04cce9 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.669361 17157 tablet_replica.cc:333] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.669965 17157 raft_consensus.cc:2243] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.670836 17157 raft_consensus.cc:2272] T 58101c530cc743ab98b421b33f16ff72 P 931f2597e137468ea1700bfa37832616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.673719 17157 tablet_replica.cc:333] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.674343 17157 raft_consensus.cc:2243] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.675614 17157 raft_consensus.cc:2272] T d2889961de7345ee9e307539e41a5a89 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.678114 17157 tablet_replica.cc:333] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.678653 17157 raft_consensus.cc:2243] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.679523 17157 raft_consensus.cc:2272] T 3dcb1327519a4d6dba7315ab535b87b3 P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.681823 17157 tablet_replica.cc:333] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616: stopping tablet replica
I20260709 12:21:34.682431 17157 raft_consensus.cc:2243] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.682852 17157 raft_consensus.cc:2272] T 22170366e5214d469b9f37d4c7c7c30f P 931f2597e137468ea1700bfa37832616 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:34.713959 28824 consensus_peers.cc:597] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:34.731360 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
W20260709 12:21:34.748099 28824 consensus_peers.cc:597] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:33699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:34.779868 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:21:34.810911 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:34.811849 17157 tablet_replica.cc:333] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.813038 17157 raft_consensus.cc:2243] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.814119 17157 raft_consensus.cc:2272] T 1e1531185db849d8bcb282065ffaa5c6 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.816471 17157 tablet_replica.cc:333] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.817054 17157 raft_consensus.cc:2243] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.817543 17157 raft_consensus.cc:2272] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.819916 17157 tablet_replica.cc:333] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.820426 17157 raft_consensus.cc:2243] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.820883 17157 raft_consensus.cc:2272] T 5d0fefc8bf46465ea8fd628770cd4ba4 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.823362 17157 tablet_replica.cc:333] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.823930 17157 raft_consensus.cc:2243] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.824833 17157 raft_consensus.cc:2272] T fb77d6947fd44daf919b6e8af52861e7 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.827068 17157 tablet_replica.cc:333] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.827644 17157 raft_consensus.cc:2243] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.828109 17157 raft_consensus.cc:2272] T dd4c0144eb8346fa8f1080616ea29109 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.830304 17157 tablet_replica.cc:333] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.830876 17157 raft_consensus.cc:2243] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.831823 17157 raft_consensus.cc:2272] T 1b575713ccdc48bfb6616b5e1afc0c60 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.834019 17157 tablet_replica.cc:333] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.834692 17157 raft_consensus.cc:2243] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.835167 17157 raft_consensus.cc:2272] T 488a5535ca71476db152cd1b9dfe5347 P d1945ee0314e465a875344905594c858 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.837388 17157 tablet_replica.cc:333] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.837885 17157 raft_consensus.cc:2243] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.838409 17157 raft_consensus.cc:2272] T 02d033d6a3fe4eb19439c707068a7b10 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.840524 17157 tablet_replica.cc:333] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.841035 17157 raft_consensus.cc:2243] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.842000 17157 raft_consensus.cc:2272] T 58fdc9b9c1cc431897d0f1242d1fdccf P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.844192 17157 tablet_replica.cc:333] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.844708 17157 raft_consensus.cc:2243] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.845196 17157 raft_consensus.cc:2272] T 091ceec6e2ad4b22be7737dcaf361b63 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.847402 17157 tablet_replica.cc:333] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.847873 17157 raft_consensus.cc:2243] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.848848 17157 raft_consensus.cc:2272] T 17e3fa379b7d4e20ad78550c233c2389 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.851140 17157 tablet_replica.cc:333] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.851689 17157 raft_consensus.cc:2243] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:34.852511 17157 raft_consensus.cc:2272] T 3ce98f1bed204d93ad21d02af807727c P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.854656 17157 tablet_replica.cc:333] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.855140 17157 raft_consensus.cc:2243] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.855551 17157 raft_consensus.cc:2272] T 0e543962a89c490a9572dc8a6ef7c792 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.857901 17157 tablet_replica.cc:333] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.858358 17157 raft_consensus.cc:2243] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.858786 17157 raft_consensus.cc:2272] T 88c811c463e64eff91a0ed703c04cce9 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.861155 17157 tablet_replica.cc:333] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.861604 17157 raft_consensus.cc:2243] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.862047 17157 raft_consensus.cc:2272] T 22170366e5214d469b9f37d4c7c7c30f P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.864230 17157 tablet_replica.cc:333] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.864737 17157 raft_consensus.cc:2243] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.865156 17157 raft_consensus.cc:2272] T 58101c530cc743ab98b421b33f16ff72 P d1945ee0314e465a875344905594c858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.867322 17157 tablet_replica.cc:333] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.867760 17157 raft_consensus.cc:2243] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.868209 17157 raft_consensus.cc:2272] T d2889961de7345ee9e307539e41a5a89 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:34.870424 17157 tablet_replica.cc:333] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858: stopping tablet replica
I20260709 12:21:34.870837 17157 raft_consensus.cc:2243] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:34.871308 17157 raft_consensus.cc:2272] T 3dcb1327519a4d6dba7315ab535b87b3 P d1945ee0314e465a875344905594c858 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:34.907872 28824 consensus_peers.cc:597] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:33699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:34.933676 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:21:34.950907 28824 consensus_peers.cc:597] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:33699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:34.974280 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:21:35.010696 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:35.011698 17157 tablet_replica.cc:333] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.012885 17157 raft_consensus.cc:2243] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.014093 17157 raft_consensus.cc:2272] T 5d0fefc8bf46465ea8fd628770cd4ba4 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:35.014106 28824 consensus_peers.cc:597] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 -> Peer 931f2597e137468ea1700bfa37832616 (127.16.193.65:33699): Couldn't send request to peer 931f2597e137468ea1700bfa37832616. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:33699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:35.016517 17157 tablet_replica.cc:333] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.017234 17157 raft_consensus.cc:2243] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.018049 17157 raft_consensus.cc:2272] T 489cf4ccbbeb4458ab1d3ca94f1f89ed P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.020256 17157 tablet_replica.cc:333] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.020798 17157 raft_consensus.cc:2243] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.021286 17157 raft_consensus.cc:2272] T 1e1531185db849d8bcb282065ffaa5c6 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.023597 17157 tablet_replica.cc:333] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.024421 17157 raft_consensus.cc:2243] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.025019 17157 raft_consensus.cc:2272] T 1b575713ccdc48bfb6616b5e1afc0c60 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.028223 17157 tablet_replica.cc:333] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
W20260709 12:21:35.028703 28827 consensus_peers.cc:597] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 -> Peer d1945ee0314e465a875344905594c858 (127.16.193.66:45499): Couldn't send request to peer d1945ee0314e465a875344905594c858. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.66:45499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:35.028949 17157 raft_consensus.cc:2243] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.029451 17157 raft_consensus.cc:2272] T dd4c0144eb8346fa8f1080616ea29109 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.031683 17157 tablet_replica.cc:333] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.032209 17157 raft_consensus.cc:2243] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.032624 17157 raft_consensus.cc:2272] T fb77d6947fd44daf919b6e8af52861e7 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.034534 17157 tablet_replica.cc:333] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.035023 17157 raft_consensus.cc:2243] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.035815 17157 raft_consensus.cc:2272] T 488a5535ca71476db152cd1b9dfe5347 P 8538ce0a884c4a10ae31c81b33103c07 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.037770 17157 tablet_replica.cc:333] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.038306 17157 raft_consensus.cc:2243] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.038733 17157 raft_consensus.cc:2272] T 02d033d6a3fe4eb19439c707068a7b10 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.040849 17157 tablet_replica.cc:333] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.041357 17157 raft_consensus.cc:2243] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.041735 17157 raft_consensus.cc:2272] T 58fdc9b9c1cc431897d0f1242d1fdccf P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.043835 17157 tablet_replica.cc:333] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.044337 17157 raft_consensus.cc:2243] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.045123 17157 raft_consensus.cc:2272] T 091ceec6e2ad4b22be7737dcaf361b63 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.047201 17157 tablet_replica.cc:333] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.047788 17157 raft_consensus.cc:2243] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.048390 17157 raft_consensus.cc:2272] T 17e3fa379b7d4e20ad78550c233c2389 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.050647 17157 tablet_replica.cc:333] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.051258 17157 raft_consensus.cc:2243] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.051767 17157 raft_consensus.cc:2272] T 3ce98f1bed204d93ad21d02af807727c P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.053742 17157 tablet_replica.cc:333] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.054267 17157 raft_consensus.cc:2243] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.054698 17157 raft_consensus.cc:2272] T 0e543962a89c490a9572dc8a6ef7c792 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.056712 17157 tablet_replica.cc:333] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.057199 17157 raft_consensus.cc:2243] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.058028 17157 raft_consensus.cc:2272] T 88c811c463e64eff91a0ed703c04cce9 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.060348 17157 tablet_replica.cc:333] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.060838 17157 raft_consensus.cc:2243] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.061308 17157 raft_consensus.cc:2272] T 3dcb1327519a4d6dba7315ab535b87b3 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.063553 17157 tablet_replica.cc:333] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.064066 17157 raft_consensus.cc:2243] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.064971 17157 raft_consensus.cc:2272] T 22170366e5214d469b9f37d4c7c7c30f P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.067453 17157 tablet_replica.cc:333] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.068008 17157 raft_consensus.cc:2243] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.068513 17157 raft_consensus.cc:2272] T d2889961de7345ee9e307539e41a5a89 P 8538ce0a884c4a10ae31c81b33103c07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.070809 17157 tablet_replica.cc:333] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07: stopping tablet replica
I20260709 12:21:35.071318 17157 raft_consensus.cc:2243] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:35.071798 17157 raft_consensus.cc:2272] T 58101c530cc743ab98b421b33f16ff72 P 8538ce0a884c4a10ae31c81b33103c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.112907 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:21:35.151290 17157 master.cc:562] Master@127.16.193.126:36829 shutting down...
I20260709 12:21:35.174243 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:35.174998 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:35.175424 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P b78b1e4c52dc481d9ba2740dc73b2277: stopping tablet replica
I20260709 12:21:35.197080 17157 master.cc:584] Master@127.16.193.126:36829 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (20931 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:21:35.251737 17157 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.193.126:34249
I20260709 12:21:35.252869 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:35.258587 29197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:35.259480 29198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.260272 17157 server_base.cc:1061] running on GCE node
W20260709 12:21:35.261161 29200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.262018 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:35.262205 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:35.262380 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599695262365 us; error 0 us; skew 500 ppm
I20260709 12:21:35.262917 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:35.265430 17157 webserver.cc:533] Webserver started at http://127.16.193.126:39019/ using document root <none> and password file <none>
I20260709 12:21:35.265928 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:35.266110 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:35.266397 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:35.268054 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/master-0-root/instance:
uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.272352 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:35.275503 29205 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.276226 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:21:35.276504 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/master-0-root
uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.276782 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:35.293839 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:35.294991 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:35.332093 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.126:34249
I20260709 12:21:35.332242 29256 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.126:34249 every 8 connection(s)
I20260709 12:21:35.336033 29257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.348111 29257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: Bootstrap starting.
I20260709 12:21:35.352872 29257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.357263 29257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: No bootstrap required, opened a new log
I20260709 12:21:35.359602 29257 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER }
I20260709 12:21:35.360066 29257 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:35.360304 29257 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5b5e25fdcb42aaa5387c3a0a5676b4, State: Initialized, Role: FOLLOWER
I20260709 12:21:35.360996 29257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER }
I20260709 12:21:35.361480 29257 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:35.361711 29257 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:35.362005 29257 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:35.366876 29257 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER }
I20260709 12:21:35.367419 29257 leader_election.cc:304] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c5b5e25fdcb42aaa5387c3a0a5676b4; no voters: 
I20260709 12:21:35.368479 29257 leader_election.cc:290] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:35.368913 29260 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:35.370301 29260 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 1 LEADER]: Becoming Leader. State: Replica: 7c5b5e25fdcb42aaa5387c3a0a5676b4, State: Running, Role: LEADER
I20260709 12:21:35.370995 29260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER }
I20260709 12:21:35.371749 29257 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:35.376566 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c5b5e25fdcb42aaa5387c3a0a5676b4. Latest consensus state: current_term: 1 leader_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER } }
I20260709 12:21:35.377403 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:35.378166 29261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5b5e25fdcb42aaa5387c3a0a5676b4" member_type: VOTER } }
I20260709 12:21:35.378815 29261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:35.379987 29268 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:35.386641 29268 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:35.389688 17157 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:35.395592 29268 catalog_manager.cc:1383] Generated new cluster ID: 71c8c3a29551466fafb919cd1954355e
I20260709 12:21:35.395867 29268 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:35.410490 29268 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:35.411821 29268 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:35.423653 29268 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: Generated new TSK 0
I20260709 12:21:35.424293 29268 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:35.456664 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:35.462795 29278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:35.464003 29279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.470191 17157 server_base.cc:1061] running on GCE node
W20260709 12:21:35.470103 29281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.471705 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:35.471905 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:35.472030 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599695472019 us; error 0 us; skew 500 ppm
I20260709 12:21:35.472491 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:35.475040 17157 webserver.cc:533] Webserver started at http://127.16.193.65:37075/ using document root <none> and password file <none>
I20260709 12:21:35.475575 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:35.475747 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:35.475989 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:35.477082 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-0-root/instance:
uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.481737 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:35.485313 29286 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.486135 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:35.486462 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-0-root
uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.486750 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:35.504011 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:35.505215 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:35.506659 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:35.508900 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:35.509068 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.509361 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:35.509533 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.548553 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.65:40545
I20260709 12:21:35.548702 29348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.65:40545 every 8 connection(s)
I20260709 12:21:35.553828 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:35.561357 29354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:35.561443 29353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:35.564826 29356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.565336 17157 server_base.cc:1061] running on GCE node
I20260709 12:21:35.566792 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:35.567055 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:35.567305 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599695567287 us; error 0 us; skew 500 ppm
I20260709 12:21:35.567989 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:35.570812 29349 heartbeater.cc:344] Connected to a master server at 127.16.193.126:34249
I20260709 12:21:35.571002 17157 webserver.cc:533] Webserver started at http://127.16.193.66:32783/ using document root <none> and password file <none>
I20260709 12:21:35.571210 29349 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:35.571825 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:35.572090 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:35.572220 29349 heartbeater.cc:507] Master 127.16.193.126:34249 requested a full tablet report, sending...
I20260709 12:21:35.572446 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:35.573999 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-1-root/instance:
uuid: "8457574bb2ea439fa9c42cda14d14a6b"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.574833 29222 ts_manager.cc:194] Registered new tserver with Master: a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545)
I20260709 12:21:35.576763 29222 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43782
I20260709 12:21:35.578928 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:21:35.582504 29361 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.583309 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:35.583611 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-1-root
uuid: "8457574bb2ea439fa9c42cda14d14a6b"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.583906 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:35.614460 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:35.615772 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:35.617643 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:35.619823 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:35.620059 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.620324 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:35.620496 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.659955 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.66:42599
I20260709 12:21:35.660055 29423 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.66:42599 every 8 connection(s)
I20260709 12:21:35.664973 17157 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:35.673226 29427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:35.675110 29428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.676787 17157 server_base.cc:1061] running on GCE node
W20260709 12:21:35.678193 29430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:35.678417 29424 heartbeater.cc:344] Connected to a master server at 127.16.193.126:34249
I20260709 12:21:35.678988 29424 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:35.679404 17157 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:35.679715 17157 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:35.679831 29424 heartbeater.cc:507] Master 127.16.193.126:34249 requested a full tablet report, sending...
I20260709 12:21:35.679900 17157 hybrid_clock.cc:648] HybridClock initialized: now 1783599695679868 us; error 0 us; skew 500 ppm
I20260709 12:21:35.680789 17157 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:35.682319 29222 ts_manager.cc:194] Registered new tserver with Master: 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:35.683459 17157 webserver.cc:533] Webserver started at http://127.16.193.67:44499/ using document root <none> and password file <none>
I20260709 12:21:35.683951 17157 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:35.684106 17157 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:35.684000 29222 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43786
I20260709 12:21:35.684505 17157 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:35.685878 17157 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-2-root/instance:
uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.690407 17157 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:35.693572 29435 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.694345 17157 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:35.694626 17157 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-2-root
uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
format_stamp: "Formatted at 2026-07-09 12:21:35 on dist-test-slave-n640"
I20260709 12:21:35.694921 17157 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskX5JRpn/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599312340324-17157-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:35.719347 17157 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:35.720562 17157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:35.722023 17157 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:35.724321 17157 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:35.724525 17157 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.724800 17157 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:35.724967 17157 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:35.764091 17157 rpc_server.cc:307] RPC server started. Bound to: 127.16.193.67:33641
I20260709 12:21:35.764206 29497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.193.67:33641 every 8 connection(s)
I20260709 12:21:35.779285 29498 heartbeater.cc:344] Connected to a master server at 127.16.193.126:34249
I20260709 12:21:35.779721 29498 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:35.780481 29498 heartbeater.cc:507] Master 127.16.193.126:34249 requested a full tablet report, sending...
I20260709 12:21:35.782423 29222 ts_manager.cc:194] Registered new tserver with Master: d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:35.783340 17157 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015657435s
I20260709 12:21:35.783921 29222 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43790
I20260709 12:21:35.808086 29222 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43806:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:35.810858 29222 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:35.878821 29463 tablet_service.cc:1511] Processing CreateTablet for tablet c9673f2a3de04ae78ee87b03e8e6e7e2 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:35.880889 29463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9673f2a3de04ae78ee87b03e8e6e7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.884395 29462 tablet_service.cc:1511] Processing CreateTablet for tablet 600f9682a067439f8959c1272aa64920 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:35.885895 29462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600f9682a067439f8959c1272aa64920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.888046 29461 tablet_service.cc:1511] Processing CreateTablet for tablet 6146c6d185674376855062ba856140db (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:35.889513 29461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146c6d185674376855062ba856140db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.911527 29460 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb11ba7ed054b2a8d8c54e65c09683b (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:35.913093 29460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb11ba7ed054b2a8d8c54e65c09683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.913504 29388 tablet_service.cc:1511] Processing CreateTablet for tablet 600f9682a067439f8959c1272aa64920 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:35.914736 29388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600f9682a067439f8959c1272aa64920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.917495 29314 tablet_service.cc:1511] Processing CreateTablet for tablet c9673f2a3de04ae78ee87b03e8e6e7e2 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:35.917891 29313 tablet_service.cc:1511] Processing CreateTablet for tablet 600f9682a067439f8959c1272aa64920 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:35.918902 29314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9673f2a3de04ae78ee87b03e8e6e7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.919539 29313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 600f9682a067439f8959c1272aa64920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.920106 29312 tablet_service.cc:1511] Processing CreateTablet for tablet 6146c6d185674376855062ba856140db (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:35.920392 29387 tablet_service.cc:1511] Processing CreateTablet for tablet 6146c6d185674376855062ba856140db (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:35.921485 29312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146c6d185674376855062ba856140db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.921844 29387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146c6d185674376855062ba856140db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.924417 29386 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb11ba7ed054b2a8d8c54e65c09683b (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:35.925804 29386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb11ba7ed054b2a8d8c54e65c09683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.925572 29310 tablet_service.cc:1511] Processing CreateTablet for tablet 67f3135f4bf74024910653e5eddc4c62 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:35.926931 29310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3135f4bf74024910653e5eddc4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.911736 29389 tablet_service.cc:1511] Processing CreateTablet for tablet c9673f2a3de04ae78ee87b03e8e6e7e2 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:35.924425 29311 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb11ba7ed054b2a8d8c54e65c09683b (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:35.928664 29389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9673f2a3de04ae78ee87b03e8e6e7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.929281 29311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb11ba7ed054b2a8d8c54e65c09683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.937479 29385 tablet_service.cc:1511] Processing CreateTablet for tablet 67f3135f4bf74024910653e5eddc4c62 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:35.939424 29385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3135f4bf74024910653e5eddc4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.940001 29459 tablet_service.cc:1511] Processing CreateTablet for tablet 67f3135f4bf74024910653e5eddc4c62 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:35.941416 29459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3135f4bf74024910653e5eddc4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.944531 29384 tablet_service.cc:1511] Processing CreateTablet for tablet aacfb04bda734b7d90fde9daea58c55e (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:35.947758 29384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacfb04bda734b7d90fde9daea58c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.949205 29309 tablet_service.cc:1511] Processing CreateTablet for tablet aacfb04bda734b7d90fde9daea58c55e (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:35.950567 29309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacfb04bda734b7d90fde9daea58c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:35.963728 29513 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:35.977283 29513 tablet_bootstrap.cc:654] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:35.984333 29458 tablet_service.cc:1511] Processing CreateTablet for tablet aacfb04bda734b7d90fde9daea58c55e (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:35.985849 29458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacfb04bda734b7d90fde9daea58c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.053360 29514 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.053640 29313 tablet_service.cc:1511] Processing CreateTablet for tablet a4ceaee2df5d459cb758be7b909fed91 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.056265 29313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ceaee2df5d459cb758be7b909fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.060493 29462 tablet_service.cc:1511] Processing CreateTablet for tablet a4ceaee2df5d459cb758be7b909fed91 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.061857 29513 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.062988 29515 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.063211 29513 ts_tablet_manager.cc:1403] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.004s
I20260709 12:21:36.063812 29514 tablet_bootstrap.cc:654] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.067783 29387 tablet_service.cc:1511] Processing CreateTablet for tablet d7de9e42928e47a4908555f387976a6f (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.065779 29462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ceaee2df5d459cb758be7b909fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.062196 29311 tablet_service.cc:1511] Processing CreateTablet for tablet d7de9e42928e47a4908555f387976a6f (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.069311 29387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7de9e42928e47a4908555f387976a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.066113 29386 tablet_service.cc:1511] Processing CreateTablet for tablet a4ceaee2df5d459cb758be7b909fed91 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.070942 29513 raft_consensus.cc:359] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.073968 29460 tablet_service.cc:1511] Processing CreateTablet for tablet d7de9e42928e47a4908555f387976a6f (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.075519 29311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7de9e42928e47a4908555f387976a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.076777 29388 tablet_service.cc:1511] Processing CreateTablet for tablet fe52819efe134429ad0ae95af9369b45 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.083746 29460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7de9e42928e47a4908555f387976a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.084503 29310 tablet_service.cc:1511] Processing CreateTablet for tablet fe52819efe134429ad0ae95af9369b45 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.085765 29310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe52819efe134429ad0ae95af9369b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.086144 29513 raft_consensus.cc:385] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.087483 29513 raft_consensus.cc:740] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.090924 29515 tablet_bootstrap.cc:654] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.088593 29459 tablet_service.cc:1511] Processing CreateTablet for tablet fe52819efe134429ad0ae95af9369b45 (DEFAULT_TABLE table=test-workload [id=855c366135914f38ab2cd19e00c34dff]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.092159 29459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe52819efe134429ad0ae95af9369b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.082692 29386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4ceaee2df5d459cb758be7b909fed91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.093392 29513 consensus_queue.cc:260] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.097903 29388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe52819efe134429ad0ae95af9369b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.099993 29515 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.100521 29515 ts_tablet_manager.cc:1403] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.004s
I20260709 12:21:36.103634 29515 raft_consensus.cc:359] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.104316 29515 raft_consensus.cc:385] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.104604 29515 raft_consensus.cc:740] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.105370 29515 consensus_queue.cc:260] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.123124 29513 ts_tablet_manager.cc:1434] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.059s	user 0.012s	sys 0.001s
I20260709 12:21:36.125778 29514 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.126291 29514 ts_tablet_manager.cc:1403] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.004s
I20260709 12:21:36.129087 29514 raft_consensus.cc:359] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.129782 29514 raft_consensus.cc:385] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.130102 29514 raft_consensus.cc:740] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.130967 29514 consensus_queue.cc:260] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.131642 29498 heartbeater.cc:499] Master 127.16.193.126:34249 was elected leader, sending a full tablet report...
I20260709 12:21:36.149184 29513 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.157469 29424 heartbeater.cc:499] Master 127.16.193.126:34249 was elected leader, sending a full tablet report...
I20260709 12:21:36.157979 29515 ts_tablet_manager.cc:1434] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.057s	user 0.016s	sys 0.041s
I20260709 12:21:36.158890 29513 tablet_bootstrap.cc:654] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.159075 29515 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
W20260709 12:21:36.164281 29425 tablet.cc:2406] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:36.165704 29515 tablet_bootstrap.cc:654] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.171993 29514 ts_tablet_manager.cc:1434] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.045s	user 0.017s	sys 0.013s
I20260709 12:21:36.172946 29349 heartbeater.cc:499] Master 127.16.193.126:34249 was elected leader, sending a full tablet report...
I20260709 12:21:36.179095 29514 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.179666 29513 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.183050 29513 ts_tablet_manager.cc:1403] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.034s	user 0.005s	sys 0.015s
I20260709 12:21:36.185539 29514 tablet_bootstrap.cc:654] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.186054 29513 raft_consensus.cc:359] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.186730 29513 raft_consensus.cc:385] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.187007 29513 raft_consensus.cc:740] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.180469 29515 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.188108 29515 ts_tablet_manager.cc:1403] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.003s
I20260709 12:21:36.190584 29515 raft_consensus.cc:359] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.191188 29515 raft_consensus.cc:385] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.191558 29515 raft_consensus.cc:740] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.191480 29513 consensus_queue.cc:260] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.192909 29515 consensus_queue.cc:260] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.197443 29515 ts_tablet_manager.cc:1434] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:21:36.198532 29515 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.201082 29513 ts_tablet_manager.cc:1434] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:36.202375 29513 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.206033 29515 tablet_bootstrap.cc:654] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.209929 29513 tablet_bootstrap.cc:654] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.214068 29514 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.214951 29514 ts_tablet_manager.cc:1403] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.014s
I20260709 12:21:36.218045 29515 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.218631 29515 ts_tablet_manager.cc:1403] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.013s
I20260709 12:21:36.218559 29514 raft_consensus.cc:359] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.219681 29514 raft_consensus.cc:385] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.220074 29514 raft_consensus.cc:740] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.221016 29514 consensus_queue.cc:260] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.222321 29515 raft_consensus.cc:359] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.223157 29515 raft_consensus.cc:385] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.223487 29515 raft_consensus.cc:740] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.223632 29514 ts_tablet_manager.cc:1434] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:21:36.224857 29514 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.224423 29515 consensus_queue.cc:260] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.230086 29515 ts_tablet_manager.cc:1434] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:36.231247 29515 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.232260 29514 tablet_bootstrap.cc:654] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.239625 29515 tablet_bootstrap.cc:654] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.244761 29519 raft_consensus.cc:493] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.245476 29519 raft_consensus.cc:515] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.265451 29519 leader_election.cc:290] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.270509 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.271737 29324 raft_consensus.cc:2468] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:36.273806 29438 leader_election.cc:304] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.275368 29520 raft_consensus.cc:493] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.276108 29520 raft_consensus.cc:515] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.286965 29520 leader_election.cc:290] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.287927 29519 raft_consensus.cc:2804] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.288743 29519 raft_consensus.cc:493] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.289197 29519 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.315973 29521 raft_consensus.cc:493] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.316583 29521 raft_consensus.cc:515] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.318621 29519 raft_consensus.cc:515] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.319257 29521 leader_election.cc:290] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.331535 29520 raft_consensus.cc:493] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.330560 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.332505 29520 raft_consensus.cc:515] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.332736 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.333446 29324 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.333868 29513 tablet_bootstrap.cc:492] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.334446 29513 ts_tablet_manager.cc:1403] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.132s	user 0.045s	sys 0.057s
I20260709 12:21:36.335419 29520 leader_election.cc:290] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641), a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545)
I20260709 12:21:36.336851 29515 tablet_bootstrap.cc:492] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.337386 29515 ts_tablet_manager.cc:1403] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.106s	user 0.031s	sys 0.027s
I20260709 12:21:36.337922 29514 tablet_bootstrap.cc:492] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.337826 29513 raft_consensus.cc:359] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.338618 29514 ts_tablet_manager.cc:1403] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.114s	user 0.044s	sys 0.043s
I20260709 12:21:36.338670 29513 raft_consensus.cc:385] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.339102 29513 raft_consensus.cc:740] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
W20260709 12:21:36.339818 29439 leader_election.cc:343] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.339913 29513 consensus_queue.cc:260] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.342001 29519 leader_election.cc:290] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.343076 29324 raft_consensus.cc:2468] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:36.344396 29438 leader_election.cc:304] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.345152 29514 raft_consensus.cc:359] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.345556 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:36.346060 29514 raft_consensus.cc:385] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.346778 29514 raft_consensus.cc:740] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.347565 29519 raft_consensus.cc:2804] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.344445 29515 raft_consensus.cc:359] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.348178 29519 raft_consensus.cc:697] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:36.347679 29514 consensus_queue.cc:260] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.349077 29519 consensus_queue.cc:237] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.351179 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.351886 29323 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.348181 29515 raft_consensus.cc:385] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.353350 29515 raft_consensus.cc:740] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.354110 29515 consensus_queue.cc:260] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.355034 29399 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.355628 29364 leader_election.cc:304] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.356671 29520 raft_consensus.cc:2804] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.356750 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.357097 29520 raft_consensus.cc:493] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.357473 29324 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.357511 29520 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.359163 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.359367 29364 leader_election.cc:304] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.361132 29536 raft_consensus.cc:2804] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.361524 29536 raft_consensus.cc:493] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.361847 29536 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.364099 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.364871 29473 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.365424 29399 raft_consensus.cc:2468] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:36.366688 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.367530 29472 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.368249 29520 raft_consensus.cc:515] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.370355 29398 raft_consensus.cc:2393] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a23cb2cefcbc487a9ae1dc6b201720fd in current term 1: Already voted for candidate 8457574bb2ea439fa9c42cda14d14a6b in this term.
I20260709 12:21:36.371344 29515 ts_tablet_manager.cc:1434] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.006s
I20260709 12:21:36.372617 29513 ts_tablet_manager.cc:1434] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.005s
I20260709 12:21:36.372963 29514 ts_tablet_manager.cc:1434] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:21:36.373848 29536 raft_consensus.cc:515] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.374285 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.374838 29472 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.376081 29536 leader_election.cc:290] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.377807 29515 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.373744 29513 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.380494 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.381050 29473 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:36.384567 29270 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:36.384711 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.385376 29324 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.386104 29514 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.388993 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.389633 29323 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.390174 29473 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.391631 29362 leader_election.cc:304] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.392748 29537 raft_consensus.cc:2804] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.393257 29537 raft_consensus.cc:697] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:36.398005 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.399531 29287 leader_election.cc:304] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:36.393464 29472 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.399782 29537 consensus_queue.cc:237] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.403344 29362 leader_election.cc:304] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.404726 29536 raft_consensus.cc:2804] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.406865 29513 tablet_bootstrap.cc:654] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.396934 29520 leader_election.cc:290] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641), a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545)
I20260709 12:21:36.399936 29323 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.410745 29521 raft_consensus.cc:2749] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.412613 29324 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.419724 29515 tablet_bootstrap.cc:654] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.420531 29514 tablet_bootstrap.cc:654] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.419826 29220 catalog_manager.cc:5697] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.410089 29536 raft_consensus.cc:697] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:36.425148 29536 consensus_queue.cc:237] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:36.430645 29221 catalog_manager.cc:5697] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.439541 29514 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.440092 29513 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.440150 29514 ts_tablet_manager.cc:1403] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.006s
I20260709 12:21:36.440733 29513 ts_tablet_manager.cc:1403] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.067s	user 0.018s	sys 0.009s
I20260709 12:21:36.442214 29515 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.442812 29515 ts_tablet_manager.cc:1403] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.010s
I20260709 12:21:36.444500 29514 raft_consensus.cc:359] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.445570 29514 raft_consensus.cc:385] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.445397 29513 raft_consensus.cc:359] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.446061 29514 raft_consensus.cc:740] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.446282 29513 raft_consensus.cc:385] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.446689 29513 raft_consensus.cc:740] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.446730 29515 raft_consensus.cc:359] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.447093 29514 consensus_queue.cc:260] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.447902 29515 raft_consensus.cc:385] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.448542 29515 raft_consensus.cc:740] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.449928 29514 ts_tablet_manager.cc:1434] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:36.449766 29515 consensus_queue.cc:260] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.450992 29514 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.453830 29515 ts_tablet_manager.cc:1434] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:36.454988 29515 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.459993 29514 tablet_bootstrap.cc:654] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.461835 29515 tablet_bootstrap.cc:654] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.465911 29513 consensus_queue.cc:260] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.472209 29513 ts_tablet_manager.cc:1434] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 12:21:36.473215 29513 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.474929 29515 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.475057 29514 tablet_bootstrap.cc:492] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.476315 29515 ts_tablet_manager.cc:1403] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:21:36.476383 29514 ts_tablet_manager.cc:1403] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 12:21:36.479117 29514 raft_consensus.cc:359] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.479884 29514 raft_consensus.cc:385] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.480197 29514 raft_consensus.cc:740] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.480098 29515 raft_consensus.cc:359] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.480994 29515 raft_consensus.cc:385] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.481339 29515 raft_consensus.cc:740] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.481387 29514 consensus_queue.cc:260] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.482308 29515 consensus_queue.cc:260] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.484422 29536 raft_consensus.cc:493] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.484989 29514 ts_tablet_manager.cc:1434] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:21:36.484889 29536 raft_consensus.cc:515] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.486366 29514 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.488289 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.488708 29536 leader_election.cc:290] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.489046 29324 raft_consensus.cc:2468] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.490358 29515 ts_tablet_manager.cc:1434] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:21:36.491179 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.491528 29515 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.492267 29513 tablet_bootstrap.cc:654] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:36.492898 29362 leader_election.cc:343] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.494151 29514 tablet_bootstrap.cc:654] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.494127 29364 leader_election.cc:304] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:36.495528 29536 raft_consensus.cc:2804] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.495976 29536 raft_consensus.cc:493] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.496407 29536 raft_consensus.cc:3060] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.499620 29515 tablet_bootstrap.cc:654] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.504498 29536 raft_consensus.cc:515] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.507892 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.507985 29513 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.508584 29324 raft_consensus.cc:3060] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.509050 29513 ts_tablet_manager.cc:1403] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.005s
I20260709 12:21:36.510212 29514 tablet_bootstrap.cc:492] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.510730 29514 ts_tablet_manager.cc:1403] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.003s
I20260709 12:21:36.513832 29513 raft_consensus.cc:359] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.514678 29513 raft_consensus.cc:385] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.515033 29513 raft_consensus.cc:740] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.515863 29515 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.516438 29515 ts_tablet_manager.cc:1403] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.006s
I20260709 12:21:36.517715 29324 raft_consensus.cc:2468] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.518977 29364 leader_election.cc:304] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.518895 29513 consensus_queue.cc:260] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.520164 29540 raft_consensus.cc:2804] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.520737 29540 raft_consensus.cc:697] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:36.520306 29515 raft_consensus.cc:359] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.521274 29515 raft_consensus.cc:385] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.521988 29513 ts_tablet_manager.cc:1434] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:36.521636 29540 consensus_queue.cc:237] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.523273 29513 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.526345 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
W20260709 12:21:36.527969 29362 leader_election.cc:343] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:36.528496 29536 leader_election.cc:290] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.531196 29513 tablet_bootstrap.cc:654] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.515096 29514 raft_consensus.cc:359] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.533185 29514 raft_consensus.cc:385] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.533511 29514 raft_consensus.cc:740] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.521617 29515 raft_consensus.cc:740] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.535653 29515 consensus_queue.cc:260] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.537873 29513 tablet_bootstrap.cc:492] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.538671 29513 ts_tablet_manager.cc:1403] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:21:36.535101 29514 consensus_queue.cc:260] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.537912 29515 ts_tablet_manager.cc:1434] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.021s	user 0.001s	sys 0.004s
I20260709 12:21:36.541409 29514 ts_tablet_manager.cc:1434] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:21:36.541648 29515 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.542493 29514 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.542667 29513 raft_consensus.cc:359] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.543821 29513 raft_consensus.cc:385] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.544292 29513 raft_consensus.cc:740] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.545531 29513 consensus_queue.cc:260] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.549558 29514 tablet_bootstrap.cc:654] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.552671 29513 ts_tablet_manager.cc:1434] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:21:36.553869 29513 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.556953 29514 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.557348 29514 ts_tablet_manager.cc:1403] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:21:36.560001 29514 raft_consensus.cc:359] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.560752 29514 raft_consensus.cc:385] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.561033 29514 raft_consensus.cc:740] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.561283 29513 tablet_bootstrap.cc:654] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.561848 29514 consensus_queue.cc:260] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.568593 29514 ts_tablet_manager.cc:1434] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:36.569700 29514 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.569468 29219 catalog_manager.cc:5697] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.571458 29219 catalog_manager.cc:5697] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.573421 29513 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.573961 29513 ts_tablet_manager.cc:1403] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:21:36.576171 29515 tablet_bootstrap.cc:654] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.577322 29513 raft_consensus.cc:359] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.578155 29513 raft_consensus.cc:385] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.578444 29513 raft_consensus.cc:740] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.580008 29514 tablet_bootstrap.cc:654] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.579490 29513 consensus_queue.cc:260] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.581332 29519 raft_consensus.cc:493] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.581787 29519 raft_consensus.cc:515] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.584714 29513 ts_tablet_manager.cc:1434] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:21:36.586020 29513 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.586359 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.587077 29399 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:36.588215 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.588917 29324 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:36.590466 29438 leader_election.cc:304] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.591547 29535 raft_consensus.cc:2804] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.591985 29535 raft_consensus.cc:493] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.592348 29535 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.593146 29513 tablet_bootstrap.cc:654] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.592484 29519 leader_election.cc:290] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.591516 29514 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.599772 29514 ts_tablet_manager.cc:1403] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.003s
I20260709 12:21:36.599845 29513 tablet_bootstrap.cc:492] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.600414 29513 ts_tablet_manager.cc:1403] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:21:36.603169 29513 raft_consensus.cc:359] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.603936 29513 raft_consensus.cc:385] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.604188 29513 raft_consensus.cc:740] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.605284 29535 raft_consensus.cc:515] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.606113 29515 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.606637 29515 ts_tablet_manager.cc:1403] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.003s
I20260709 12:21:36.607677 29535 leader_election.cc:290] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.603016 29514 raft_consensus.cc:359] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.608784 29514 raft_consensus.cc:385] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.609066 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.609064 29514 raft_consensus.cc:740] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.609757 29324 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.609131 29515 raft_consensus.cc:359] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.610101 29514 consensus_queue.cc:260] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.610904 29515 raft_consensus.cc:385] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.611294 29515 raft_consensus.cc:740] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.612102 29515 consensus_queue.cc:260] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.612470 29514 ts_tablet_manager.cc:1434] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:21:36.610206 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:36.613641 29399 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.614163 29515 ts_tablet_manager.cc:1434] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:36.615216 29515 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:36.619483 29514 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:36.620963 29515 tablet_bootstrap.cc:654] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.622366 29399 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:36.622714 29324 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:36.623720 29439 leader_election.cc:304] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.609380 29513 consensus_queue.cc:260] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.626174 29535 raft_consensus.cc:2804] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.628403 29535 raft_consensus.cc:697] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:36.630126 29535 consensus_queue.cc:237] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.641739 29513 ts_tablet_manager.cc:1434] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.007s
I20260709 12:21:36.643513 29513 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:36.644004 29514 tablet_bootstrap.cc:654] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.649252 29514 tablet_bootstrap.cc:492] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:36.649760 29514 ts_tablet_manager.cc:1403] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 12:21:36.651456 29513 tablet_bootstrap.cc:654] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.651634 29221 catalog_manager.cc:5697] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.652516 29514 raft_consensus.cc:359] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.653321 29514 raft_consensus.cc:385] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.653635 29514 raft_consensus.cc:740] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.654371 29514 consensus_queue.cc:260] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.656409 29514 ts_tablet_manager.cc:1434] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:21:36.658716 29515 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:36.659168 29515 ts_tablet_manager.cc:1403] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.027s
I20260709 12:21:36.661367 29513 tablet_bootstrap.cc:492] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:36.661918 29513 ts_tablet_manager.cc:1403] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.005s
I20260709 12:21:36.661991 29515 raft_consensus.cc:359] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.662742 29515 raft_consensus.cc:385] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.663036 29515 raft_consensus.cc:740] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.663859 29515 consensus_queue.cc:260] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.665091 29513 raft_consensus.cc:359] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.665863 29513 raft_consensus.cc:385] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:36.666195 29513 raft_consensus.cc:740] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:36.666929 29513 consensus_queue.cc:260] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.669126 29515 ts_tablet_manager.cc:1434] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:21:36.669178 29513 ts_tablet_manager.cc:1434] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:21:36.672966 29521 raft_consensus.cc:493] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.673513 29521 raft_consensus.cc:515] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.675340 29521 leader_election.cc:290] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.676566 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.677145 29472 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:36.678129 29287 leader_election.cc:304] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.678627 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.679486 29521 raft_consensus.cc:2804] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.679986 29521 raft_consensus.cc:493] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.679859 29399 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:36.680379 29521 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.687142 29521 raft_consensus.cc:515] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.689040 29521 leader_election.cc:290] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.689765 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:36.690045 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.690465 29399 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.690629 29472 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.697888 29472 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:36.697888 29399 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:36.699219 29290 leader_election.cc:304] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.699920 29521 raft_consensus.cc:2804] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.700727 29521 raft_consensus.cc:697] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:36.701365 29521 consensus_queue.cc:237] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.708397 29222 catalog_manager.cc:5697] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.740417 29540 raft_consensus.cc:493] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.741024 29540 raft_consensus.cc:515] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.742776 29540 leader_election.cc:290] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.743819 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4ceaee2df5d459cb758be7b909fed91" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.743932 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4ceaee2df5d459cb758be7b909fed91" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.744524 29324 raft_consensus.cc:2468] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.744611 29472 raft_consensus.cc:2468] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.745549 29364 leader_election.cc:304] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.746325 29540 raft_consensus.cc:2804] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.746624 29540 raft_consensus.cc:493] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.746923 29540 raft_consensus.cc:3060] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.752142 29540 raft_consensus.cc:515] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.753681 29540 leader_election.cc:290] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.754552 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4ceaee2df5d459cb758be7b909fed91" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.754762 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4ceaee2df5d459cb758be7b909fed91" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.755204 29324 raft_consensus.cc:3060] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.755333 29472 raft_consensus.cc:3060] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.762192 29472 raft_consensus.cc:2468] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.762192 29324 raft_consensus.cc:2468] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.763428 29364 leader_election.cc:304] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.764107 29540 raft_consensus.cc:2804] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.764521 29540 raft_consensus.cc:697] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:36.765312 29540 consensus_queue.cc:237] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.772513 29222 catalog_manager.cc:5697] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.780711 29540 consensus_queue.cc:1048] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.789714 29521 raft_consensus.cc:493] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.790292 29521 raft_consensus.cc:515] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.792383 29521 leader_election.cc:290] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.793375 29537 consensus_queue.cc:1048] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.793553 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.794281 29473 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:36.796849 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.797616 29399 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:36.797202 29287 leader_election.cc:304] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.800733 29521 raft_consensus.cc:2804] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.801141 29521 raft_consensus.cc:493] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.801658 29521 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.819561 29521 raft_consensus.cc:515] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.823851 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.823031 29521 leader_election.cc:290] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.824630 29472 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.823100 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:36.828056 29399 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.834944 29472 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:36.835716 29399 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:36.836937 29287 leader_election.cc:304] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:36.838055 29521 raft_consensus.cc:2804] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.838819 29521 raft_consensus.cc:697] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:36.839926 29521 consensus_queue.cc:237] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.849838 29219 catalog_manager.cc:5697] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.889776 29537 raft_consensus.cc:493] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.890339 29537 raft_consensus.cc:515] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.892083 29537 leader_election.cc:290] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.892953 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.893169 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:36.893759 29324 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.893759 29472 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:36.894994 29364 leader_election.cc:304] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.895766 29537 raft_consensus.cc:2804] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:36.896097 29537 raft_consensus.cc:493] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.896381 29537 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.901702 29537 raft_consensus.cc:515] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.903241 29537 leader_election.cc:290] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:36.903918 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:36.904152 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:36.904465 29324 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.904778 29472 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.909359 29324 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.910365 29364 leader_election.cc:304] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:36.910534 29472 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:36.911690 29537 raft_consensus.cc:2804] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:36.912225 29537 raft_consensus.cc:697] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:36.912950 29537 consensus_queue.cc:237] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.920703 29219 catalog_manager.cc:5697] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:36.948223 29535 consensus_queue.cc:1048] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.950277 29537 consensus_queue.cc:1048] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:36.953163 29535 raft_consensus.cc:493] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.953778 29535 raft_consensus.cc:515] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:36.957204 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:36.957747 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacfb04bda734b7d90fde9daea58c55e" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:36.958904 29324 raft_consensus.cc:2393] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4dbb6919e7e4472ad819aefad1d45a0 in current term 1: Already voted for candidate 8457574bb2ea439fa9c42cda14d14a6b in this term.
I20260709 12:21:36.960076 29438 leader_election.cc:304] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4dbb6919e7e4472ad819aefad1d45a0; no voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:36.957806 29535 leader_election.cc:290] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:36.964356 29535 raft_consensus.cc:3060] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:36.970278 29540 consensus_queue.cc:1048] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:36.972074 29535 raft_consensus.cc:2749] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:36.970366 29577 consensus_queue.cc:1048] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:36.985529 29536 consensus_queue.cc:1048] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.004979 29537 consensus_queue.cc:1048] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.049319 29519 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.059919 29577 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.083690 29399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:51044
I20260709 12:21:37.084352 29399 raft_consensus.cc:606] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:37.097585 29473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "600f9682a067439f8959c1272aa64920"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:56058
I20260709 12:21:37.098155 29473 raft_consensus.cc:606] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:37.098248 29521 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.104605 29399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6146c6d185674376855062ba856140db"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:51044
I20260709 12:21:37.105160 29399 raft_consensus.cc:606] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:37.110110 29521 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.111330 29472 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:56058
I20260709 12:21:37.112013 29472 raft_consensus.cc:606] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:37.135450 29324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
mode: GRACEFUL
new_leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:55650
I20260709 12:21:37.136013 29324 raft_consensus.cc:606] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Received request to transfer leadership to TS 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:37.146646 29324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
mode: GRACEFUL
new_leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:55650
I20260709 12:21:37.147176 29324 raft_consensus.cc:606] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Received request to transfer leadership to TS 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:37.151768 29399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
mode: GRACEFUL
new_leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:51044
I20260709 12:21:37.152267 29399 raft_consensus.cc:606] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Received request to transfer leadership to TS d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:37.168720 29537 consensus_queue.cc:1048] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.179553 29540 consensus_queue.cc:1048] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.278774 29582 raft_consensus.cc:993] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.279327 29540 raft_consensus.cc:1081] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.280807 29521 consensus_queue.cc:1048] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.281237 29324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6146c6d185674376855062ba856140db"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55636
I20260709 12:21:37.282001 29324 raft_consensus.cc:493] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.282378 29324 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.290434 29324 raft_consensus.cc:515] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.292892 29324 leader_election.cc:290] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:37.293398 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:37.294155 29399 raft_consensus.cc:3055] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.294000 29591 consensus_queue.cc:1048] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.294528 29399 raft_consensus.cc:740] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:37.295903 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146c6d185674376855062ba856140db" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.296525 29472 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.299077 29399 consensus_queue.cc:260] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.302525 29597 raft_consensus.cc:993] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: : Instructing follower 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:37.303062 29591 raft_consensus.cc:1081] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Signalling peer 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:37.303341 29399 raft_consensus.cc:3060] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.304837 29398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:51028
I20260709 12:21:37.305413 29398 raft_consensus.cc:493] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.305756 29398 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.306226 29472 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.307425 29287 leader_election.cc:304] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:37.308214 29591 raft_consensus.cc:2804] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.308692 29591 raft_consensus.cc:697] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.309484 29591 consensus_queue.cc:237] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.310480 29399 raft_consensus.cc:2468] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.316895 29398 raft_consensus.cc:515] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.318735 29398 leader_election.cc:290] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:37.319815 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:37.320114 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f3135f4bf74024910653e5eddc4c62" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.320461 29324 raft_consensus.cc:3055] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.319936 29222 catalog_manager.cc:5697] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.320753 29472 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.320848 29324 raft_consensus.cc:740] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.321810 29324 consensus_queue.cc:260] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.322957 29324 raft_consensus.cc:3060] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.326992 29472 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:37.328518 29362 leader_election.cc:304] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:37.329375 29540 raft_consensus.cc:2804] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.329762 29324 raft_consensus.cc:2468] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:37.329923 29540 raft_consensus.cc:697] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:37.330886 29540 consensus_queue.cc:237] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.337220 29222 catalog_manager.cc:5697] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 1 to 2, leader changed from a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65) to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 2 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.378849 29540 consensus_queue.cc:1048] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.389427 29582 consensus_queue.cc:1048] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.399403 29582 raft_consensus.cc:993] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: : Instructing follower d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:37.399992 29540 raft_consensus.cc:1081] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Signalling peer d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:37.401678 29472 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:21:37.402292 29472 raft_consensus.cc:493] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.402685 29472 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.408555 29472 raft_consensus.cc:515] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.410499 29472 leader_election.cc:290] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 2 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:37.411262 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:37.411648 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe52819efe134429ad0ae95af9369b45" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:37.412007 29324 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.412362 29398 raft_consensus.cc:3055] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.412756 29398 raft_consensus.cc:740] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:37.413499 29398 consensus_queue.cc:260] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.414454 29398 raft_consensus.cc:3060] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.418687 29324 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 2.
I20260709 12:21:37.419708 29398 raft_consensus.cc:2468] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 2.
I20260709 12:21:37.419999 29438 leader_election.cc:304] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:37.420845 29577 raft_consensus.cc:2804] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.421277 29577 raft_consensus.cc:697] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:37.422061 29577 consensus_queue.cc:237] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.428151 29219 catalog_manager.cc:5697] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 1 to 2, leader changed from 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66) to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 2 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.507925 29582 raft_consensus.cc:993] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.508399 29540 raft_consensus.cc:1081] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.510178 29324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55636
I20260709 12:21:37.510661 29324 raft_consensus.cc:493] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.510969 29324 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.516781 29324 raft_consensus.cc:515] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:37.518357 29324 leader_election.cc:290] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:37.519011 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.519325 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9673f2a3de04ae78ee87b03e8e6e7e2" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:37.519685 29472 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.519961 29398 raft_consensus.cc:3055] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.520308 29398 raft_consensus.cc:740] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:37.520955 29398 consensus_queue.cc:260] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:37.521903 29398 raft_consensus.cc:3060] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.524866 29472 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.526049 29287 leader_election.cc:304] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:37.526825 29591 raft_consensus.cc:2804] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.527163 29398 raft_consensus.cc:2468] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.527324 29591 raft_consensus.cc:697] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.528324 29591 consensus_queue.cc:237] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } }
I20260709 12:21:37.537416 29219 catalog_manager.cc:5697] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.538729 29519 raft_consensus.cc:993] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.539264 29577 raft_consensus.cc:1081] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.540923 29324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55630
I20260709 12:21:37.541519 29324 raft_consensus.cc:493] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.541918 29324 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.548532 29597 raft_consensus.cc:993] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: : Instructing follower 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:37.549103 29591 raft_consensus.cc:1081] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Signalling peer 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:37.551090 29398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:51028
I20260709 12:21:37.551704 29398 raft_consensus.cc:493] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.552173 29398 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.552259 29324 raft_consensus.cc:515] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.554728 29324 leader_election.cc:290] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:37.555183 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:37.555873 29399 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.558421 29398 raft_consensus.cc:515] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.555848 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.559646 29472 raft_consensus.cc:3055] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.559981 29472 raft_consensus.cc:740] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:37.560571 29398 leader_election.cc:290] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:37.561441 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:37.561338 29472 consensus_queue.cc:260] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.562136 29324 raft_consensus.cc:3055] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.562248 29399 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.562546 29324 raft_consensus.cc:740] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.563508 29290 leader_election.cc:304] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:37.562381 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.562727 29472 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.563814 29324 consensus_queue.cc:260] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.564337 29473 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.564486 29591 raft_consensus.cc:2804] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.565092 29591 raft_consensus.cc:697] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.565230 29324 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.566026 29591 consensus_queue.cc:237] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.571390 29472 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.573532 29324 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:37.574470 29473 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:37.574818 29364 leader_election.cc:304] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:37.574470 29222 catalog_manager.cc:5697] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.575842 29582 raft_consensus.cc:2804] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.576345 29582 raft_consensus.cc:697] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:37.577069 29582 consensus_queue.cc:237] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.587308 29219 catalog_manager.cc:5697] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 1 to 2, leader changed from a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65) to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 2 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.602478 29519 raft_consensus.cc:993] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.602977 29577 raft_consensus.cc:1081] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:37.604461 29324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "600f9682a067439f8959c1272aa64920"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55630
I20260709 12:21:37.604928 29324 raft_consensus.cc:493] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.605290 29324 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.609683 29324 raft_consensus.cc:515] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.611268 29324 leader_election.cc:290] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:37.612021 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:37.612399 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "600f9682a067439f8959c1272aa64920" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:37.612655 29399 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.613005 29473 raft_consensus.cc:3055] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:37.613348 29473 raft_consensus.cc:740] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:37.614027 29473 consensus_queue.cc:260] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.614989 29473 raft_consensus.cc:3060] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.617918 29399 raft_consensus.cc:2468] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.619140 29290 leader_election.cc:304] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:37.619988 29591 raft_consensus.cc:2804] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:37.620388 29473 raft_consensus.cc:2468] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:37.620438 29591 raft_consensus.cc:697] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:37.621539 29591 consensus_queue.cc:237] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:37.627907 29219 catalog_manager.cc:5697] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.714335 29473 raft_consensus.cc:1275] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.715643 29582 consensus_queue.cc:1048] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.722882 29324 raft_consensus.cc:1275] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.724309 29583 consensus_queue.cc:1048] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.802750 29324 raft_consensus.cc:1275] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.804141 29577 consensus_queue.cc:1048] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.811596 29399 raft_consensus.cc:1275] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.813616 29535 consensus_queue.cc:1048] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.881543 29473 raft_consensus.cc:1275] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.882961 29591 consensus_queue.cc:1048] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.891983 29399 raft_consensus.cc:1275] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.893319 29591 consensus_queue.cc:1048] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.928347 29473 raft_consensus.cc:1275] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.929709 29591 consensus_queue.cc:1048] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.938448 29399 raft_consensus.cc:1275] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.940166 29521 consensus_queue.cc:1048] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.972239 29473 raft_consensus.cc:1275] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.973601 29521 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.983671 29399 raft_consensus.cc:1275] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:37.985636 29591 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.026515 29473 raft_consensus.cc:1275] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.027904 29583 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.035214 29324 raft_consensus.cc:1275] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.036855 29540 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.113351 29399 raft_consensus.cc:1275] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.114697 29591 consensus_queue.cc:1048] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.124003 29473 raft_consensus.cc:1275] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.125648 29597 consensus_queue.cc:1048] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:40.179600 29221 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43810:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:40.182147 29221 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:40.231185 29310 tablet_service.cc:1511] Processing CreateTablet for tablet fff5eef9749f40118843a09a8958f788 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:40.232574 29310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff5eef9749f40118843a09a8958f788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.232317 29386 tablet_service.cc:1511] Processing CreateTablet for tablet fff5eef9749f40118843a09a8958f788 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:40.233577 29386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff5eef9749f40118843a09a8958f788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.233639 29462 tablet_service.cc:1511] Processing CreateTablet for tablet fff5eef9749f40118843a09a8958f788 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:40.234936 29462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff5eef9749f40118843a09a8958f788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.236634 29388 tablet_service.cc:1511] Processing CreateTablet for tablet f7775a7515354b7f9ef7fb11f619f128 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:40.237938 29388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7775a7515354b7f9ef7fb11f619f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.241205 29387 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca231f6e37b4c8f9b87f9b8982da72b (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:40.242486 29387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca231f6e37b4c8f9b87f9b8982da72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.244240 29311 tablet_service.cc:1511] Processing CreateTablet for tablet f7775a7515354b7f9ef7fb11f619f128 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:40.245541 29311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7775a7515354b7f9ef7fb11f619f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.246028 29385 tablet_service.cc:1511] Processing CreateTablet for tablet f0b49a8b27bd40d49c8460792ae53b55 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:40.247323 29385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b49a8b27bd40d49c8460792ae53b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.248203 29460 tablet_service.cc:1511] Processing CreateTablet for tablet f7775a7515354b7f9ef7fb11f619f128 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:40.249490 29460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7775a7515354b7f9ef7fb11f619f128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.251629 29384 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed2cb7f137e4341b90c5859de219572 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:40.252933 29384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed2cb7f137e4341b90c5859de219572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.256204 29459 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca231f6e37b4c8f9b87f9b8982da72b (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:40.257016 29389 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f8e2533c45e8becbb28502629c19 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:40.257463 29459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca231f6e37b4c8f9b87f9b8982da72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.258301 29389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f8e2533c45e8becbb28502629c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.260000 29461 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f8e2533c45e8becbb28502629c19 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:40.260427 29463 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed2cb7f137e4341b90c5859de219572 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:40.261742 29461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f8e2533c45e8becbb28502629c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.262002 29458 tablet_service.cc:1511] Processing CreateTablet for tablet f0b49a8b27bd40d49c8460792ae53b55 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:40.268666 29313 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca231f6e37b4c8f9b87f9b8982da72b (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:40.270018 29313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca231f6e37b4c8f9b87f9b8982da72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.269652 29309 tablet_service.cc:1511] Processing CreateTablet for tablet f0b49a8b27bd40d49c8460792ae53b55 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:40.271034 29309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b49a8b27bd40d49c8460792ae53b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.277112 29308 tablet_service.cc:1511] Processing CreateTablet for tablet f4a56e02dd2f4d6694005ea175134f6a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:40.278383 29308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a56e02dd2f4d6694005ea175134f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.272330 29312 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed2cb7f137e4341b90c5859de219572 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:40.282173 29312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed2cb7f137e4341b90c5859de219572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.285086 29457 tablet_service.cc:1511] Processing CreateTablet for tablet f4a56e02dd2f4d6694005ea175134f6a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:40.287580 29307 tablet_service.cc:1511] Processing CreateTablet for tablet 892f6fa502274e7fb35cc13423c31191 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:40.273890 29314 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb4f8e2533c45e8becbb28502629c19 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:40.288820 29307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892f6fa502274e7fb35cc13423c31191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.289391 29314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb4f8e2533c45e8becbb28502629c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.289765 29382 tablet_service.cc:1511] Processing CreateTablet for tablet 892f6fa502274e7fb35cc13423c31191 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:40.275418 29458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b49a8b27bd40d49c8460792ae53b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.291085 29382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892f6fa502274e7fb35cc13423c31191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.280717 29383 tablet_service.cc:1511] Processing CreateTablet for tablet f4a56e02dd2f4d6694005ea175134f6a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:40.293269 29383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a56e02dd2f4d6694005ea175134f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.294265 29456 tablet_service.cc:1511] Processing CreateTablet for tablet 892f6fa502274e7fb35cc13423c31191 (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:40.294557 29306 tablet_service.cc:1511] Processing CreateTablet for tablet 88521e06c94842ee86e176f041a1762a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:40.295524 29463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed2cb7f137e4341b90c5859de219572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.295816 29306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88521e06c94842ee86e176f041a1762a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.298996 29456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892f6fa502274e7fb35cc13423c31191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.303580 29457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a56e02dd2f4d6694005ea175134f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.306733 29381 tablet_service.cc:1511] Processing CreateTablet for tablet 88521e06c94842ee86e176f041a1762a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:40.308130 29381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88521e06c94842ee86e176f041a1762a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:40.314662 29455 tablet_service.cc:1511] Processing CreateTablet for tablet 88521e06c94842ee86e176f041a1762a (DEFAULT_TABLE table=soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:40.316035 29455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88521e06c94842ee86e176f041a1762a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:40.432639 29270 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:40.456017 29619 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.463553 29620 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.476593 29619 tablet_bootstrap.cc:654] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.472332 29621 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.483887 29620 tablet_bootstrap.cc:654] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.484817 29619 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.485330 29619 ts_tablet_manager.cc:1403] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.003s
I20260709 12:21:40.486115 29621 tablet_bootstrap.cc:654] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.487752 29619 raft_consensus.cc:359] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.488987 29619 raft_consensus.cc:385] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.489357 29619 raft_consensus.cc:740] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.490139 29619 consensus_queue.cc:260] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.490693 29620 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.492241 29619 ts_tablet_manager.cc:1434] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:40.492419 29621 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.492960 29621 ts_tablet_manager.cc:1403] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.009s
I20260709 12:21:40.493201 29619 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.494496 29620 ts_tablet_manager.cc:1403] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.006s
I20260709 12:21:40.496107 29621 raft_consensus.cc:359] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.496893 29621 raft_consensus.cc:385] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.497188 29621 raft_consensus.cc:740] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.498004 29620 raft_consensus.cc:359] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.497984 29621 consensus_queue.cc:260] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.500032 29619 tablet_bootstrap.cc:654] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.500598 29621 ts_tablet_manager.cc:1434] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:40.501704 29621 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.506361 29620 raft_consensus.cc:385] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.506785 29620 raft_consensus.cc:740] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.507602 29620 consensus_queue.cc:260] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.508460 29621 tablet_bootstrap.cc:654] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.510571 29620 ts_tablet_manager.cc:1434] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:21:40.511888 29620 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.521194 29620 tablet_bootstrap.cc:654] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.522238 29619 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.522782 29619 ts_tablet_manager.cc:1403] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.003s
I20260709 12:21:40.526198 29621 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.526954 29621 ts_tablet_manager.cc:1403] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.018s
I20260709 12:21:40.526825 29619 raft_consensus.cc:359] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.527742 29619 raft_consensus.cc:385] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.528065 29619 raft_consensus.cc:740] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.528692 29619 consensus_queue.cc:260] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.530433 29619 ts_tablet_manager.cc:1434] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:40.530326 29621 raft_consensus.cc:359] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.531178 29621 raft_consensus.cc:385] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.531533 29621 raft_consensus.cc:740] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.531570 29619 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.532217 29621 consensus_queue.cc:260] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.534354 29621 ts_tablet_manager.cc:1434] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:40.535410 29621 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.537802 29619 tablet_bootstrap.cc:654] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.542585 29621 tablet_bootstrap.cc:654] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.551659 29620 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.552328 29620 ts_tablet_manager.cc:1403] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.004s
I20260709 12:21:40.554250 29621 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.554804 29621 ts_tablet_manager.cc:1403] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.002s
I20260709 12:21:40.556243 29620 raft_consensus.cc:359] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.557340 29620 raft_consensus.cc:385] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.557710 29620 raft_consensus.cc:740] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.557934 29621 raft_consensus.cc:359] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.558835 29621 raft_consensus.cc:385] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.559211 29621 raft_consensus.cc:740] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.558681 29620 consensus_queue.cc:260] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.559999 29621 consensus_queue.cc:260] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.562143 29620 ts_tablet_manager.cc:1434] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:21:40.562386 29621 ts_tablet_manager.cc:1434] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 12:21:40.563488 29621 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.564586 29620 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.565868 29619 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.566380 29619 ts_tablet_manager.cc:1403] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.011s
I20260709 12:21:40.569489 29619 raft_consensus.cc:359] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.570137 29621 tablet_bootstrap.cc:654] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.570223 29619 raft_consensus.cc:385] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.570719 29619 raft_consensus.cc:740] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.571573 29620 tablet_bootstrap.cc:654] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.571566 29619 consensus_queue.cc:260] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.582983 29621 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.583535 29621 ts_tablet_manager.cc:1403] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.002s
I20260709 12:21:40.584335 29620 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.584451 29619 ts_tablet_manager.cc:1434] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:21:40.585500 29619 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.585909 29620 ts_tablet_manager.cc:1403] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 12:21:40.586774 29621 raft_consensus.cc:359] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.587586 29621 raft_consensus.cc:385] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.587881 29621 raft_consensus.cc:740] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.588778 29621 consensus_queue.cc:260] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.588994 29620 raft_consensus.cc:359] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.589839 29620 raft_consensus.cc:385] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.590147 29620 raft_consensus.cc:740] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.591336 29621 ts_tablet_manager.cc:1434] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:40.591050 29620 consensus_queue.cc:260] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.592470 29621 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.593501 29620 ts_tablet_manager.cc:1434] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:40.594585 29620 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.599954 29621 tablet_bootstrap.cc:654] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.601373 29620 tablet_bootstrap.cc:654] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.602926 29619 tablet_bootstrap.cc:654] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.616990 29620 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.617681 29620 ts_tablet_manager.cc:1403] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:21:40.621093 29620 raft_consensus.cc:359] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.621805 29620 raft_consensus.cc:385] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.622092 29620 raft_consensus.cc:740] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.622844 29620 consensus_queue.cc:260] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.624522 29620 ts_tablet_manager.cc:1434] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 12:21:40.626087 29620 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.632671 29620 tablet_bootstrap.cc:654] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.641348 29619 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.641983 29619 ts_tablet_manager.cc:1403] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.014s
I20260709 12:21:40.644070 29620 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.644595 29620 ts_tablet_manager.cc:1403] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 12:21:40.645162 29619 raft_consensus.cc:359] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.645959 29619 raft_consensus.cc:385] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.646243 29619 raft_consensus.cc:740] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.646637 29540 raft_consensus.cc:493] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.647020 29619 consensus_queue.cc:260] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.647601 29620 raft_consensus.cc:359] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.648344 29620 raft_consensus.cc:385] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.648658 29620 raft_consensus.cc:740] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.647743 29540 raft_consensus.cc:515] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.649775 29619 ts_tablet_manager.cc:1434] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:21:40.649432 29620 consensus_queue.cc:260] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.652411 29619 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.653774 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f8e2533c45e8becbb28502629c19" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
W20260709 12:21:40.655318 29364 leader_election.cc:343] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.656314 29540 leader_election.cc:290] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
W20260709 12:21:40.638664 29362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.67:33641, user_credentials={real_user=slave}} blocked reactor thread for 47635.3us
I20260709 12:21:40.657845 29620 ts_tablet_manager.cc:1434] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:40.657936 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f8e2533c45e8becbb28502629c19" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:40.658598 29472 raft_consensus.cc:2468] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:40.658958 29620 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.660321 29362 leader_election.cc:304] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:40.662443 29540 raft_consensus.cc:2804] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.662815 29540 raft_consensus.cc:493] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.662989 29619 tablet_bootstrap.cc:654] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.663203 29540 raft_consensus.cc:3060] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.669937 29620 tablet_bootstrap.cc:654] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.677505 29597 raft_consensus.cc:493] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.678215 29597 raft_consensus.cc:515] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.681717 29621 tablet_bootstrap.cc:492] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.682111 29597 leader_election.cc:290] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.682412 29621 ts_tablet_manager.cc:1403] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.090s	user 0.046s	sys 0.028s
I20260709 12:21:40.682369 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:40.683271 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:40.684049 29472 raft_consensus.cc:2468] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
W20260709 12:21:40.685364 29290 leader_election.cc:343] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.686170 29287 leader_election.cc:304] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:40.687017 29619 tablet_bootstrap.cc:492] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.687745 29619 ts_tablet_manager.cc:1403] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.004s
I20260709 12:21:40.688509 29621 raft_consensus.cc:359] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.689216 29597 raft_consensus.cc:2804] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.689544 29621 raft_consensus.cc:385] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.689713 29597 raft_consensus.cc:493] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.690073 29621 raft_consensus.cc:740] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.691495 29621 consensus_queue.cc:260] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.690209 29597 raft_consensus.cc:3060] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.691833 29540 raft_consensus.cc:515] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.698995 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f8e2533c45e8becbb28502629c19" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:40.698720 29619 raft_consensus.cc:359] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.699864 29472 raft_consensus.cc:3060] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.700323 29619 raft_consensus.cc:385] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.700803 29619 raft_consensus.cc:740] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.705340 29619 consensus_queue.cc:260] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.699752 29540 leader_election.cc:290] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.710809 29540 raft_consensus.cc:493] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.711306 29540 raft_consensus.cc:515] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.714314 29540 leader_election.cc:290] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.715703 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
W20260709 12:21:40.717538 29362 leader_election.cc:343] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.717327 29597 raft_consensus.cc:515] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.718680 29472 raft_consensus.cc:2468] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:40.721323 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:40.721719 29597 leader_election.cc:290] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.722515 29362 leader_election.cc:304] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:40.725006 29620 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.725661 29620 ts_tablet_manager.cc:1403] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.022s
I20260709 12:21:40.726413 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:40.727205 29472 raft_consensus.cc:3060] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.728147 29621 ts_tablet_manager.cc:1434] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.043s	user 0.012s	sys 0.004s
I20260709 12:21:40.727121 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb4f8e2533c45e8becbb28502629c19" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
W20260709 12:21:40.731317 29290 leader_election.cc:343] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.727733 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:40.733441 29323 raft_consensus.cc:2468] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:40.734483 29621 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.736112 29472 raft_consensus.cc:2468] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:40.738961 29287 leader_election.cc:304] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:40.739655 29639 raft_consensus.cc:2804] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.740918 29639 raft_consensus.cc:697] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
W20260709 12:21:40.737775 29364 leader_election.cc:343] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.739813 29597 raft_consensus.cc:2804] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.741812 29639 consensus_queue.cc:237] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.745718 29364 leader_election.cc:304] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:40.736905 29620 raft_consensus.cc:359] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.748140 29620 raft_consensus.cc:385] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.748238 29632 raft_consensus.cc:2804] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.748490 29620 raft_consensus.cc:740] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.748708 29632 raft_consensus.cc:493] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.749063 29632 raft_consensus.cc:3060] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.749336 29620 consensus_queue.cc:260] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.763388 29597 raft_consensus.cc:697] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:40.768059 29620 ts_tablet_manager.cc:1434] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.006s
I20260709 12:21:40.771555 29620 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.770471 29619 ts_tablet_manager.cc:1434] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.082s	user 0.019s	sys 0.015s
I20260709 12:21:40.774461 29621 tablet_bootstrap.cc:654] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.769912 29220 catalog_manager.cc:5697] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.769656 29632 raft_consensus.cc:515] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.778735 29619 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.780789 29632 leader_election.cc:290] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.782191 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:40.769534 29597 consensus_queue.cc:237] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.782994 29323 raft_consensus.cc:3060] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.783036 29620 tablet_bootstrap.cc:654] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.793001 29619 tablet_bootstrap.cc:654] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.795766 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
W20260709 12:21:40.798026 29362 leader_election.cc:343] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.803217 29219 catalog_manager.cc:5697] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.807713 29642 raft_consensus.cc:493] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.808270 29642 raft_consensus.cc:515] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.809614 29643 raft_consensus.cc:493] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.809882 29620 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.810448 29620 ts_tablet_manager.cc:1403] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.008s
I20260709 12:21:40.810316 29643 raft_consensus.cc:515] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.811878 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:40.812582 29472 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:40.813767 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:40.818086 29619 tablet_bootstrap.cc:492] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.813683 29620 raft_consensus.cc:359] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.815404 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:40.818804 29287 leader_election.cc:304] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:40.810673 29642 leader_election.cc:290] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.818645 29619 ts_tablet_manager.cc:1403] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.013s
I20260709 12:21:40.818951 29620 raft_consensus.cc:385] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.816160 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:40.819415 29399 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:40.818778 29323 raft_consensus.cc:2468] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
W20260709 12:21:40.822441 29287 leader_election.cc:343] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.819854 29642 raft_consensus.cc:2804] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.823910 29642 raft_consensus.cc:493] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.824276 29642 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.823432 29364 leader_election.cc:304] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:40.825059 29619 raft_consensus.cc:359] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.826045 29619 raft_consensus.cc:385] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.826191 29640 raft_consensus.cc:2804] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.826411 29619 raft_consensus.cc:740] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.826766 29640 raft_consensus.cc:697] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:40.827335 29619 consensus_queue.cc:260] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.827847 29640 consensus_queue.cc:237] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.829704 29619 ts_tablet_manager.cc:1434] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:21:40.830765 29619 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.821944 29620 raft_consensus.cc:740] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
W20260709 12:21:40.831562 29290 leader_election.cc:343] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:40.832084 29290 leader_election.cc:304] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:40.832007 29620 consensus_queue.cc:260] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.833262 29597 raft_consensus.cc:2749] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.817011 29643 leader_election.cc:290] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.835265 29642 raft_consensus.cc:515] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.840941 29642 leader_election.cc:290] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.841646 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:40.842186 29398 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.843050 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:40.843716 29473 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.849474 29398 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:40.850715 29290 leader_election.cc:304] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:40.851111 29473 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:40.852175 29621 tablet_bootstrap.cc:492] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.851778 29642 raft_consensus.cc:2804] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.852710 29621 ts_tablet_manager.cc:1403] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.004s
I20260709 12:21:40.852710 29642 raft_consensus.cc:697] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:40.854884 29642 consensus_queue.cc:237] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.858340 29619 tablet_bootstrap.cc:654] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.863742 29620 ts_tablet_manager.cc:1434] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.053s	user 0.019s	sys 0.018s
I20260709 12:21:40.864965 29620 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.871258 29621 raft_consensus.cc:359] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.872301 29621 raft_consensus.cc:385] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.872680 29621 raft_consensus.cc:740] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.873137 29620 tablet_bootstrap.cc:654] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.873909 29621 consensus_queue.cc:260] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.878221 29621 ts_tablet_manager.cc:1434] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 12:21:40.879232 29621 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.888473 29619 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.888855 29620 tablet_bootstrap.cc:492] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.889123 29619 ts_tablet_manager.cc:1403] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.009s
I20260709 12:21:40.889300 29620 ts_tablet_manager.cc:1403] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 12:21:40.889505 29621 tablet_bootstrap.cc:654] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.891912 29620 raft_consensus.cc:359] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.892774 29620 raft_consensus.cc:385] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.893044 29620 raft_consensus.cc:740] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.893757 29620 consensus_queue.cc:260] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.896525 29620 ts_tablet_manager.cc:1434] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:40.897605 29620 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: Bootstrap starting.
I20260709 12:21:40.902426 29621 tablet_bootstrap.cc:492] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.902938 29621 ts_tablet_manager.cc:1403] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 12:21:40.903697 29620 tablet_bootstrap.cc:654] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.906139 29621 raft_consensus.cc:359] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.906884 29621 raft_consensus.cc:385] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.907191 29621 raft_consensus.cc:740] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.907948 29621 consensus_queue.cc:260] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.910331 29621 ts_tablet_manager.cc:1434] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:40.911331 29621 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.911917 29640 raft_consensus.cc:493] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.912489 29640 raft_consensus.cc:515] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.911764 29219 catalog_manager.cc:5697] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.915405 29641 raft_consensus.cc:493] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.915959 29641 raft_consensus.cc:515] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.917233 29620 tablet_bootstrap.cc:492] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: No bootstrap required, opened a new log
I20260709 12:21:40.917613 29620 ts_tablet_manager.cc:1403] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.001s
I20260709 12:21:40.918184 29621 tablet_bootstrap.cc:654] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.918220 29641 leader_election.cc:290] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:40.919601 29640 leader_election.cc:290] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.919842 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:40.919947 29620 raft_consensus.cc:359] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.920691 29620 raft_consensus.cc:385] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.920945 29620 raft_consensus.cc:740] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.921139 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:40.921766 29473 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:40.921625 29620 consensus_queue.cc:260] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.921819 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:40.922616 29398 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:40.923374 29362 leader_election.cc:304] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:40.923775 29620 ts_tablet_manager.cc:1434] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:21:40.923928 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:40.924556 29323 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:40.926234 29439 leader_election.cc:304] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:40.927408 29641 raft_consensus.cc:2804] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.927747 29641 raft_consensus.cc:493] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.928062 29641 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.929991 29220 catalog_manager.cc:5697] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:40.932880 29621 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.927857 29640 raft_consensus.cc:2804] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:40.933377 29621 ts_tablet_manager.cc:1403] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.009s
I20260709 12:21:40.933353 29640 raft_consensus.cc:493] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.933935 29640 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.934476 29641 raft_consensus.cc:515] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.936734 29619 raft_consensus.cc:359] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.937316 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:40.937388 29641 leader_election.cc:290] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:40.937594 29619 raft_consensus.cc:385] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.938072 29323 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:40.938201 29619 raft_consensus.cc:740] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.938877 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:40.939411 29619 consensus_queue.cc:260] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.940047 29621 raft_consensus.cc:359] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.940742 29621 raft_consensus.cc:385] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.941002 29621 raft_consensus.cc:740] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.941696 29621 consensus_queue.cc:260] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.942075 29619 ts_tablet_manager.cc:1434] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 12:21:40.943338 29619 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:40.943764 29621 ts_tablet_manager.cc:1434] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:40.944758 29621 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: Bootstrap starting.
I20260709 12:21:40.945650 29323 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:40.946853 29438 leader_election.cc:304] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:40.948204 29606 raft_consensus.cc:2804] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:40.948654 29606 raft_consensus.cc:697] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:40.949462 29606 consensus_queue.cc:237] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.950067 29619 tablet_bootstrap.cc:654] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.950476 29621 tablet_bootstrap.cc:654] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:40.962421 29621 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: No bootstrap required, opened a new log
I20260709 12:21:40.962903 29621 ts_tablet_manager.cc:1403] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 12:21:40.966076 29621 raft_consensus.cc:359] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.966801 29621 raft_consensus.cc:385] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.967070 29621 raft_consensus.cc:740] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.967870 29621 consensus_queue.cc:260] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.969981 29621 ts_tablet_manager.cc:1434] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:40.988251 29640 raft_consensus.cc:515] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.990342 29398 raft_consensus.cc:2393] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4dbb6919e7e4472ad819aefad1d45a0 in current term 1: Already voted for candidate 8457574bb2ea439fa9c42cda14d14a6b in this term.
I20260709 12:21:40.993507 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:40.993534 29640 leader_election.cc:290] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:40.993748 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:40.994277 29324 raft_consensus.cc:2393] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8457574bb2ea439fa9c42cda14d14a6b in current term 1: Already voted for candidate d4dbb6919e7e4472ad819aefad1d45a0 in this term.
I20260709 12:21:40.995167 29619 tablet_bootstrap.cc:492] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:40.995584 29619 ts_tablet_manager.cc:1403] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.026s
I20260709 12:21:40.996948 29364 leader_election.cc:304] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b; no voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:40.997797 29640 raft_consensus.cc:2749] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:40.997681 29619 raft_consensus.cc:359] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.998450 29619 raft_consensus.cc:385] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:40.998736 29619 raft_consensus.cc:740] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:40.999372 29619 consensus_queue.cc:260] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:40.999681 29221 catalog_manager.cc:5697] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:41.001312 29619 ts_tablet_manager.cc:1434] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:41.001644 29641 raft_consensus.cc:493] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.002267 29619 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: Bootstrap starting.
I20260709 12:21:41.002099 29641 raft_consensus.cc:515] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.004189 29641 leader_election.cc:290] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:41.005661 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:41.006247 29398 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:41.007014 29439 leader_election.cc:304] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.007858 29606 raft_consensus.cc:2804] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.008071 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:41.008255 29606 raft_consensus.cc:493] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.008697 29324 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:41.008833 29619 tablet_bootstrap.cc:654] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:41.010504 29606 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.016633 29606 raft_consensus.cc:515] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.017382 29619 tablet_bootstrap.cc:492] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: No bootstrap required, opened a new log
I20260709 12:21:41.017895 29619 ts_tablet_manager.cc:1403] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 12:21:41.018901 29606 leader_election.cc:290] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:41.019481 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:41.020011 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:41.020052 29398 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.020349 29619 raft_consensus.cc:359] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.020767 29324 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.021044 29619 raft_consensus.cc:385] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:41.021401 29619 raft_consensus.cc:740] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Initialized, Role: FOLLOWER
I20260709 12:21:41.022066 29619 consensus_queue.cc:260] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.024116 29619 ts_tablet_manager.cc:1434] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:41.027446 29324 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:41.027447 29398 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:41.028677 29438 leader_election.cc:304] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.029459 29606 raft_consensus.cc:2804] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.029961 29606 raft_consensus.cc:697] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:41.030802 29606 consensus_queue.cc:237] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.037846 29221 catalog_manager.cc:5697] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:41.056752 29640 raft_consensus.cc:493] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.057359 29640 raft_consensus.cc:515] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.059124 29640 leader_election.cc:290] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.059978 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:41.060156 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:41.060544 29324 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:41.060839 29473 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 0.
I20260709 12:21:41.061520 29364 leader_election.cc:304] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:41.062359 29640 raft_consensus.cc:2804] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.062625 29640 raft_consensus.cc:493] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.062839 29640 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.067885 29640 raft_consensus.cc:515] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.069397 29640 leader_election.cc:290] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.070238 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:41.070825 29324 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.070727 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:41.071853 29473 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.075919 29324 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:41.076845 29364 leader_election.cc:304] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:41.076864 29473 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 1.
I20260709 12:21:41.077632 29640 raft_consensus.cc:2804] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.078159 29640 raft_consensus.cc:697] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:41.078963 29640 consensus_queue.cc:237] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.086946 29221 catalog_manager.cc:5697] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 1 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.171653 29606 raft_consensus.cc:493] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.172242 29606 raft_consensus.cc:515] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.174068 29606 leader_election.cc:290] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:41.174897 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:41.175084 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:41.175494 29324 raft_consensus.cc:2468] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:41.175796 29398 raft_consensus.cc:2468] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 0.
I20260709 12:21:41.176496 29438 leader_election.cc:304] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.177356 29606 raft_consensus.cc:2804] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.177723 29606 raft_consensus.cc:493] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.178112 29606 raft_consensus.cc:3060] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.183765 29606 raft_consensus.cc:515] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.185391 29606 leader_election.cc:290] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:41.186285 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:41.186486 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88521e06c94842ee86e176f041a1762a" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:41.186827 29324 raft_consensus.cc:3060] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.187201 29398 raft_consensus.cc:3060] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.191609 29324 raft_consensus.cc:2468] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:41.192543 29398 raft_consensus.cc:2468] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 1.
I20260709 12:21:41.192541 29438 leader_election.cc:304] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.193516 29606 raft_consensus.cc:2804] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.193970 29606 raft_consensus.cc:697] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:41.194754 29606 consensus_queue.cc:237] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.201982 29221 catalog_manager.cc:5697] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 1 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:41.221573 29640 raft_consensus.cc:493] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.221961 29640 raft_consensus.cc:515] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.223848 29640 leader_election.cc:290] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.224442 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:41.224860 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fff5eef9749f40118843a09a8958f788" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:41.225680 29473 raft_consensus.cc:2393] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8457574bb2ea439fa9c42cda14d14a6b in current term 1: Already voted for candidate a23cb2cefcbc487a9ae1dc6b201720fd in this term.
I20260709 12:21:41.226606 29362 leader_election.cc:304] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b; no voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:41.227315 29640 raft_consensus.cc:3060] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.232195 29640 raft_consensus.cc:2749] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.247758 29324 raft_consensus.cc:3060] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.249250 29606 raft_consensus.cc:493] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.249614 29642 consensus_queue.cc:1048] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.249922 29606 raft_consensus.cc:515] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.252777 29323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" is_pre_election: true
I20260709 12:21:41.253108 29606 leader_election.cc:290] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:41.253554 29323 raft_consensus.cc:2393] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4dbb6919e7e4472ad819aefad1d45a0 in current term 1: Already voted for candidate 8457574bb2ea439fa9c42cda14d14a6b in this term.
I20260709 12:21:41.253604 29642 raft_consensus.cc:493] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.254221 29642 raft_consensus.cc:515] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.254765 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892f6fa502274e7fb35cc13423c31191" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:41.256485 29439 leader_election.cc:304] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4dbb6919e7e4472ad819aefad1d45a0; no voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:41.256501 29642 leader_election.cc:290] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.257896 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7775a7515354b7f9ef7fb11f619f128" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" is_pre_election: true
I20260709 12:21:41.258087 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7775a7515354b7f9ef7fb11f619f128" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b" is_pre_election: true
I20260709 12:21:41.258615 29606 raft_consensus.cc:3060] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.258796 29399 raft_consensus.cc:2468] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:41.259114 29640 consensus_queue.cc:1048] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.258615 29473 raft_consensus.cc:2468] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 0.
I20260709 12:21:41.260550 29287 leader_election.cc:304] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.261359 29642 raft_consensus.cc:2804] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:41.261726 29642 raft_consensus.cc:493] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.262056 29642 raft_consensus.cc:3060] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.272867 29606 raft_consensus.cc:2749] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:41.281359 29643 consensus_queue.cc:1048] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.283030 29642 raft_consensus.cc:515] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.283529 29639 consensus_queue.cc:1048] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.286094 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7775a7515354b7f9ef7fb11f619f128" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:41.286605 29472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7775a7515354b7f9ef7fb11f619f128" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:41.286803 29399 raft_consensus.cc:3060] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.287293 29472 raft_consensus.cc:3060] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:41.295411 29472 raft_consensus.cc:2468] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:41.295941 29399 raft_consensus.cc:2468] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 1.
I20260709 12:21:41.296797 29287 leader_election.cc:304] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.297679 29597 raft_consensus.cc:2804] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:41.298195 29597 raft_consensus.cc:697] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:41.299115 29597 consensus_queue.cc:237] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.304087 29642 leader_election.cc:290] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 1 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.318694 29221 catalog_manager.cc:5697] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 1 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.335652 29597 consensus_queue.cc:1048] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.349260 29597 consensus_queue.cc:1048] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.363749 29640 consensus_queue.cc:1048] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.376461 29640 consensus_queue.cc:1048] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.476617 29606 consensus_queue.cc:1048] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.487963 29606 consensus_queue.cc:1048] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:41.518229 29399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:51044
I20260709 12:21:41.518745 29399 raft_consensus.cc:606] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:41.523605 29473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
mode: GRACEFUL
new_leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:56058
I20260709 12:21:41.524171 29473 raft_consensus.cc:606] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Received request to transfer leadership to TS a23cb2cefcbc487a9ae1dc6b201720fd
I20260709 12:21:41.529367 29473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
mode: GRACEFUL
new_leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:56058
I20260709 12:21:41.529901 29473 raft_consensus.cc:606] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Received request to transfer leadership to TS 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:41.537606 29324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
mode: GRACEFUL
new_leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:55650
I20260709 12:21:41.538249 29324 raft_consensus.cc:606] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Received request to transfer leadership to TS 8457574bb2ea439fa9c42cda14d14a6b
I20260709 12:21:41.539700 29640 consensus_queue.cc:1048] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.546871 29632 raft_consensus.cc:993] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:41.547358 29632 raft_consensus.cc:1081] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:41.549392 29324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55636
I20260709 12:21:41.549957 29324 raft_consensus.cc:493] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.550367 29324 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.552850 29606 consensus_queue.cc:1048] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.554955 29640 consensus_queue.cc:1048] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.556619 29324 raft_consensus.cc:515] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.559315 29399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:41.559839 29399 raft_consensus.cc:3055] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.559883 29323 raft_consensus.cc:1240] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Rejecting Update request from peer 8457574bb2ea439fa9c42cda14d14a6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.560225 29399 raft_consensus.cc:740] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:41.561173 29399 consensus_queue.cc:260] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.562038 29399 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.566282 29682 consensus_queue.cc:1048] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.568195 29324 leader_election.cc:290] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.568310 29399 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:41.569504 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca231f6e37b4c8f9b87f9b8982da72b" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:41.569967 29473 raft_consensus.cc:3060] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.570039 29290 leader_election.cc:304] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:41.570935 29597 raft_consensus.cc:2804] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:41.571422 29597 raft_consensus.cc:697] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:41.572225 29597 consensus_queue.cc:237] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.578265 29473 raft_consensus.cc:2468] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:41.585078 29682 raft_consensus.cc:993] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: : Instructing follower 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:41.585518 29682 raft_consensus.cc:1081] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Signalling peer 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:41.588377 29398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:51020
I20260709 12:21:41.590044 29398 raft_consensus.cc:493] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.590420 29398 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.592136 29220 catalog_manager.cc:5697] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.597368 29398 raft_consensus.cc:515] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.602974 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:41.603612 29473 raft_consensus.cc:3055] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.603946 29473 raft_consensus.cc:740] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:41.604000 29322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed2cb7f137e4341b90c5859de219572" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:41.604668 29322 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.604795 29473 consensus_queue.cc:260] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.606026 29473 raft_consensus.cc:3060] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.611070 29322 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:41.611732 29398 leader_election.cc:290] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.612396 29364 leader_election.cc:304] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:41.613243 29640 raft_consensus.cc:2804] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:41.613739 29640 raft_consensus.cc:697] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:41.614539 29640 consensus_queue.cc:237] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.616597 29473 raft_consensus.cc:2468] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:41.623974 29222 catalog_manager.cc:5697] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 1 to 2, leader changed from d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67) to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 2 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.624768 29606 consensus_queue.cc:1048] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.634824 29682 consensus_queue.cc:1048] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.649142 29642 raft_consensus.cc:993] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: : Instructing follower 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:41.649622 29642 raft_consensus.cc:1081] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Signalling peer 8457574bb2ea439fa9c42cda14d14a6b to start an election
I20260709 12:21:41.651175 29398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
 from {username='slave'} at 127.0.0.1:51028
I20260709 12:21:41.651777 29398 raft_consensus.cc:493] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.652159 29398 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.658768 29398 raft_consensus.cc:515] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.660948 29398 leader_election.cc:290] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:41.661276 29322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:41.661538 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4a56e02dd2f4d6694005ea175134f6a" candidate_uuid: "8457574bb2ea439fa9c42cda14d14a6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:41.661932 29322 raft_consensus.cc:3055] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.662220 29473 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.662343 29322 raft_consensus.cc:740] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:41.663220 29322 consensus_queue.cc:260] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.664268 29322 raft_consensus.cc:3060] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.668622 29473 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:41.669471 29322 raft_consensus.cc:2468] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8457574bb2ea439fa9c42cda14d14a6b in term 2.
I20260709 12:21:41.669669 29362 leader_election.cc:304] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:41.670573 29640 raft_consensus.cc:2804] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:41.671063 29640 raft_consensus.cc:697] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Becoming Leader. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:41.671813 29640 consensus_queue.cc:237] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:41.678551 29221 catalog_manager.cc:5697] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b reported cstate change: term changed from 1 to 2, leader changed from a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65) to 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66). New cstate: current_term: 2 leader_uuid: "8457574bb2ea439fa9c42cda14d14a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:41.689518 29597 consensus_queue.cc:1048] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:41.699841 29643 consensus_queue.cc:1048] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:42.020776 29641 raft_consensus.cc:993] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: : Instructing follower a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:42.021220 29682 raft_consensus.cc:1081] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Signalling peer a23cb2cefcbc487a9ae1dc6b201720fd to start an election
I20260709 12:21:42.022753 29322 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
 from {username='slave'} at 127.0.0.1:55630
I20260709 12:21:42.023185 29322 raft_consensus.cc:493] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.023520 29322 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.026834 29324 raft_consensus.cc:1275] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.028496 29639 consensus_queue.cc:1048] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.029449 29322 raft_consensus.cc:515] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:42.031586 29322 leader_election.cc:290] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Requested vote from peers 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599), d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67:33641)
I20260709 12:21:42.033004 29398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:42.034296 29398 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.037438 29471 raft_consensus.cc:1275] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.039101 29639 consensus_queue.cc:1048] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.042515 29398 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:42.044929 29398 raft_consensus.cc:1240] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Rejecting Update request from peer d4dbb6919e7e4472ad819aefad1d45a0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:42.045403 29290 leader_election.cc:304] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8457574bb2ea439fa9c42cda14d14a6b, a23cb2cefcbc487a9ae1dc6b201720fd; no voters: 
I20260709 12:21:42.046365 29641 consensus_queue.cc:1059] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.046380 29643 raft_consensus.cc:2804] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:42.047456 29682 raft_consensus.cc:3055] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:42.047523 29473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b49a8b27bd40d49c8460792ae53b55" candidate_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
I20260709 12:21:42.047950 29682 raft_consensus.cc:740] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:42.051868 29682 consensus_queue.cc:260] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:42.052990 29682 raft_consensus.cc:3060] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.048010 29643 raft_consensus.cc:697] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Leader. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:42.055912 29643 consensus_queue.cc:237] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:42.065088 29220 catalog_manager.cc:5697] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd reported cstate change: term changed from 1 to 2, leader changed from d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67) to a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65). New cstate: current_term: 2 leader_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.069146 29473 raft_consensus.cc:2468] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a23cb2cefcbc487a9ae1dc6b201720fd in term 2.
I20260709 12:21:42.081020 29398 raft_consensus.cc:1275] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.082427 29643 consensus_queue.cc:1048] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.090911 29473 raft_consensus.cc:1275] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.092548 29597 consensus_queue.cc:1048] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.165884 29324 raft_consensus.cc:1275] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.167166 29639 consensus_queue.cc:1048] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.175109 29471 raft_consensus.cc:1275] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer 8457574bb2ea439fa9c42cda14d14a6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.177397 29640 consensus_queue.cc:1048] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.475765 29398 raft_consensus.cc:1275] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.476982 29643 consensus_queue.cc:1048] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.485973 29473 raft_consensus.cc:1275] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Refusing update from remote peer a23cb2cefcbc487a9ae1dc6b201720fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.487915 29643 consensus_queue.cc:1048] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:44.553573 29220 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43810:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:21:44.578290 29220 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43810:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:21:44.712930 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:44.763623 29718 log.cc:927] Time spent T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:21:44.820883 29740 tablet.cc:1724] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:44.844101 29739 tablet.cc:1724] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:44.867211 29731 log.cc:927] Time spent T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.002s
I20260709 12:21:44.887514 29742 tablet.cc:1724] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:44.891989 29743 tablet.cc:1724] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:44.910218 29720 log.cc:927] Time spent T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Append to log took a long time: real 0.148s	user 0.000s	sys 0.002s
W20260709 12:21:44.930716 29734 log.cc:927] Time spent T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 12:21:44.942154 29750 tablet.cc:1724] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:44.948148 29747 tablet.cc:1724] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:44.963415 29728 log.cc:927] Time spent T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:21:44.964112 29753 tablet.cc:1724] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:44.964270 29748 tablet.cc:1724] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:44.973204 29752 tablet.cc:1724] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:44.992799 29746 log.cc:927] Time spent T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:21:45.003115 29740 tablet.cc:1724] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.038486 29743 tablet.cc:1724] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.041216 29758 tablet.cc:1724] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.054132 29750 tablet.cc:1724] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.058692 29739 tablet.cc:1724] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.070183 29747 tablet.cc:1724] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.063881 29760 tablet.cc:1724] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.073230 29757 tablet.cc:1724] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.068861 29743 tablet.cc:1724] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.071127 29742 tablet.cc:1724] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.123287 29320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b"
dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
mode: GRACEFUL
new_leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:54106
I20260709 12:21:45.124192 29320 raft_consensus.cc:606] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Received request to transfer leadership to TS d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:45.125104 29748 tablet.cc:1724] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.157135 29752 tablet.cc:1724] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.158411 29760 tablet.cc:1724] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.155977 29753 tablet.cc:1724] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.178352 29740 tablet.cc:1724] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.167222 29758 tablet.cc:1724] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.172267 29747 tablet.cc:1724] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:45.199895 29439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.66:42599, user_credentials={real_user=slave}} blocked reactor thread for 60510.3us
W20260709 12:21:45.202069 29438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.193.65:40545, user_credentials={real_user=slave}} blocked reactor thread for 63976.1us
I20260709 12:21:45.204551 29739 tablet.cc:1724] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:45.236599 29393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f"
dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
mode: GRACEFUL
new_leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:21:45.237345 29393 raft_consensus.cc:606] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Received request to transfer leadership to TS d4dbb6919e7e4472ad819aefad1d45a0
I20260709 12:21:45.238894 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:45.239470 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:45.244477 29221 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: soft_deleted_table [id=d3fe76ee7be5405ca8dccd482ced1299] alter complete (version 1)
I20260709 12:21:45.472222 29729 raft_consensus.cc:993] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: : Instructing follower d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:45.472980 29732 raft_consensus.cc:1081] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Signalling peer d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:45.474889 29471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:56052
I20260709 12:21:45.475612 29471 raft_consensus.cc:493] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:45.475967 29471 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.482999 29471 raft_consensus.cc:515] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.486090 29471 leader_election.cc:290] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 3 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:45.486438 29727 raft_consensus.cc:993] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: : Instructing follower d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:45.486274 29320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:45.487324 29320 raft_consensus.cc:3055] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:45.487818 29320 raft_consensus.cc:740] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a23cb2cefcbc487a9ae1dc6b201720fd, State: Running, Role: LEADER
I20260709 12:21:45.487136 29727 raft_consensus.cc:1081] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Signalling peer d4dbb6919e7e4472ad819aefad1d45a0 to start an election
I20260709 12:21:45.488955 29395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb11ba7ed054b2a8d8c54e65c09683b" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:45.489094 29320 consensus_queue.cc:260] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.489711 29395 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.490108 29471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f"
dest_uuid: "d4dbb6919e7e4472ad819aefad1d45a0"
 from {username='slave'} at 127.0.0.1:56040
I20260709 12:21:45.490728 29471 raft_consensus.cc:493] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:45.490948 29320 raft_consensus.cc:3060] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.491149 29471 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.499212 29471 raft_consensus.cc:515] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.500061 29395 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 3.
I20260709 12:21:45.500679 29320 raft_consensus.cc:2468] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 3.
I20260709 12:21:45.501040 29471 leader_election.cc:290] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 3 election: Requested vote from peers a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545), 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66:42599)
I20260709 12:21:45.502158 29324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd"
I20260709 12:21:45.502528 29393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7de9e42928e47a4908555f387976a6f" candidate_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8457574bb2ea439fa9c42cda14d14a6b"
I20260709 12:21:45.502908 29324 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.503248 29393 raft_consensus.cc:3055] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:45.502907 29438 leader_election.cc:304] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:45.503859 29393 raft_consensus.cc:740] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8457574bb2ea439fa9c42cda14d14a6b, State: Running, Role: LEADER
I20260709 12:21:45.504561 29764 raft_consensus.cc:2804] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:45.505113 29764 raft_consensus.cc:697] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:45.504879 29393 consensus_queue.cc:260] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.506337 29393 raft_consensus.cc:3060] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.506188 29764 consensus_queue.cc:237] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.511188 29324 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 3.
I20260709 12:21:45.512640 29438 leader_election.cc:304] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a23cb2cefcbc487a9ae1dc6b201720fd, d4dbb6919e7e4472ad819aefad1d45a0; no voters: 
I20260709 12:21:45.513562 29764 raft_consensus.cc:2804] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:45.514379 29764 raft_consensus.cc:697] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 LEADER]: Becoming Leader. State: Replica: d4dbb6919e7e4472ad819aefad1d45a0, State: Running, Role: LEADER
I20260709 12:21:45.515339 29764 consensus_queue.cc:237] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } }
I20260709 12:21:45.518368 29393 raft_consensus.cc:2468] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4dbb6919e7e4472ad819aefad1d45a0 in term 3.
I20260709 12:21:45.520437 29222 catalog_manager.cc:5697] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 2 to 3, leader changed from a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65) to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:45.533063 29222 catalog_manager.cc:5697] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 reported cstate change: term changed from 2 to 3, leader changed from 8457574bb2ea439fa9c42cda14d14a6b (127.16.193.66) to d4dbb6919e7e4472ad819aefad1d45a0 (127.16.193.67). New cstate: current_term: 3 leader_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4dbb6919e7e4472ad819aefad1d45a0" member_type: VOTER last_known_addr { host: "127.16.193.67" port: 33641 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:45.939639 29324 raft_consensus.cc:1275] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:45.941027 29764 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:45.951774 29395 raft_consensus.cc:1275] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:45.953601 29682 consensus_queue.cc:1048] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:45.982683 29395 raft_consensus.cc:1275] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:45.983853 29766 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8457574bb2ea439fa9c42cda14d14a6b" member_type: VOTER last_known_addr { host: "127.16.193.66" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:45.992949 29324 raft_consensus.cc:1275] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Refusing update from remote peer d4dbb6919e7e4472ad819aefad1d45a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:45.994843 29682 consensus_queue.cc:1048] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23cb2cefcbc487a9ae1dc6b201720fd" member_type: VOTER last_known_addr { host: "127.16.193.65" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:46.240525 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:46.245484 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:46.245988 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:47.247032 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:47.252038 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:47.252394 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:48.253311 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:48.258361 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:48.258782 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:49.259816 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:49.265763 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:49.266161 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:50.267050 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:50.272015 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:50.272415 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:51.273545 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:51.279328 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:51.279737 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:52.280712 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:52.286718 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:52.287120 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:53.288000 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:53.293107 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:53.293516 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:54.294694 17157 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:54.301391 17157 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:54.302004 17157 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:55.380795 17157 tablet_server.cc:179] TabletServer@127.16.193.65:0 shutting down...
W20260709 12:21:55.411716 29438 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:21:55.416813 29438 consensus_peers.cc:597] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.423964 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:55.425175 17157 tablet_replica.cc:333] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.426676 17157 raft_consensus.cc:2243] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.427676 17157 raft_consensus.cc:2272] T 9cb4f8e2533c45e8becbb28502629c19 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.430685 17157 tablet_replica.cc:333] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.431557 17157 raft_consensus.cc:2243] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.432518 17157 raft_consensus.cc:2272] T 1ca231f6e37b4c8f9b87f9b8982da72b P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.435315 17157 tablet_replica.cc:333] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.436004 17157 raft_consensus.cc:2243] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.436586 17157 raft_consensus.cc:2272] T 892f6fa502274e7fb35cc13423c31191 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.439164 17157 tablet_replica.cc:333] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.439915 17157 raft_consensus.cc:2243] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:21:55.440331 29253 debug-util.cc:398] Leaking SignalData structure 0x7b080008ed60 after lost signal to thread 17160
I20260709 12:21:55.441257 17157 raft_consensus.cc:2272] T f0b49a8b27bd40d49c8460792ae53b55 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:55.441558 29253 debug-util.cc:398] Leaking SignalData structure 0x7b0800177da0 after lost signal to thread 29256
W20260709 12:21:55.442917 29253 debug-util.cc:398] Leaking SignalData structure 0x7b080023f7a0 after lost signal to thread 29348
W20260709 12:21:55.444126 29253 debug-util.cc:398] Leaking SignalData structure 0x7b08002c6ae0 after lost signal to thread 29423
I20260709 12:21:55.444512 17157 tablet_replica.cc:333] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
W20260709 12:21:55.445241 29253 debug-util.cc:398] Leaking SignalData structure 0x7b08001182c0 after lost signal to thread 29497
I20260709 12:21:55.445254 17157 raft_consensus.cc:2243] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.446713 17157 raft_consensus.cc:2272] T 600f9682a067439f8959c1272aa64920 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.449707 17157 tablet_replica.cc:333] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.450398 17157 raft_consensus.cc:2243] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.451067 17157 raft_consensus.cc:2272] T 88521e06c94842ee86e176f041a1762a P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.454012 17157 tablet_replica.cc:333] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.454658 17157 raft_consensus.cc:2243] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.455745 17157 raft_consensus.cc:2272] T f7775a7515354b7f9ef7fb11f619f128 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.458626 17157 tablet_replica.cc:333] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.459388 17157 raft_consensus.cc:2243] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.459950 17157 raft_consensus.cc:2272] T fe52819efe134429ad0ae95af9369b45 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.462903 17157 tablet_replica.cc:333] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.463549 17157 raft_consensus.cc:2243] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.464113 17157 raft_consensus.cc:2272] T 67f3135f4bf74024910653e5eddc4c62 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.466843 17157 tablet_replica.cc:333] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.467478 17157 raft_consensus.cc:2243] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.468648 17157 raft_consensus.cc:2272] T fff5eef9749f40118843a09a8958f788 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.472544 17157 tablet_replica.cc:333] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.473310 17157 raft_consensus.cc:2243] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.474079 17157 raft_consensus.cc:2272] T 9cb11ba7ed054b2a8d8c54e65c09683b P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:55.474889 29364 consensus_peers.cc:597] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.477376 17157 tablet_replica.cc:333] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.478137 17157 raft_consensus.cc:2243] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.479403 17157 raft_consensus.cc:2272] T c9673f2a3de04ae78ee87b03e8e6e7e2 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.482282 17157 tablet_replica.cc:333] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.482916 17157 raft_consensus.cc:2243] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.483566 17157 raft_consensus.cc:2272] T aacfb04bda734b7d90fde9daea58c55e P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.487443 17157 tablet_replica.cc:333] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.488147 17157 raft_consensus.cc:2243] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.489322 17157 raft_consensus.cc:2272] T 6146c6d185674376855062ba856140db P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.492115 17157 tablet_replica.cc:333] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.492695 17157 raft_consensus.cc:2243] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.493219 17157 raft_consensus.cc:2272] T d7de9e42928e47a4908555f387976a6f P a23cb2cefcbc487a9ae1dc6b201720fd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.495913 17157 tablet_replica.cc:333] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.496526 17157 raft_consensus.cc:2243] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.497200 17157 raft_consensus.cc:2272] T 6ed2cb7f137e4341b90c5859de219572 P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.499910 17157 tablet_replica.cc:333] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.500494 17157 raft_consensus.cc:2243] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.501072 17157 raft_consensus.cc:2272] T f4a56e02dd2f4d6694005ea175134f6a P a23cb2cefcbc487a9ae1dc6b201720fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.503953 17157 tablet_replica.cc:333] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd: stopping tablet replica
I20260709 12:21:55.504415 17157 raft_consensus.cc:2243] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.504902 17157 raft_consensus.cc:2272] T a4ceaee2df5d459cb758be7b909fed91 P a23cb2cefcbc487a9ae1dc6b201720fd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:55.519634 29364 consensus_peers.cc:597] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:55.556109 29438 consensus_peers.cc:597] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.561197 17157 tablet_server.cc:196] TabletServer@127.16.193.65:0 shutdown complete.
W20260709 12:21:55.594242 29364 consensus_peers.cc:597] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.630735 17157 tablet_server.cc:179] TabletServer@127.16.193.66:0 shutting down...
I20260709 12:21:55.669706 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:55.670611 17157 tablet_replica.cc:333] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.671587 17157 raft_consensus.cc:2243] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.672300 17157 raft_consensus.cc:2272] T fff5eef9749f40118843a09a8958f788 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.675658 17157 tablet_replica.cc:333] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.676644 17157 raft_consensus.cc:2243] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.677901 17157 raft_consensus.cc:2272] T 9cb4f8e2533c45e8becbb28502629c19 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.681787 17157 tablet_replica.cc:333] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.682655 17157 raft_consensus.cc:2243] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.683274 17157 raft_consensus.cc:2272] T f7775a7515354b7f9ef7fb11f619f128 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.686219 17157 tablet_replica.cc:333] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.686751 17157 raft_consensus.cc:2243] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.687419 17157 raft_consensus.cc:2272] T fe52819efe134429ad0ae95af9369b45 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.690533 17157 tablet_replica.cc:333] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.691404 17157 raft_consensus.cc:2243] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.692672 17157 raft_consensus.cc:2272] T 67f3135f4bf74024910653e5eddc4c62 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:55.695003 29364 consensus_peers.cc:597] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.695910 17157 tablet_replica.cc:333] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.696664 17157 raft_consensus.cc:2243] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.697564 17157 raft_consensus.cc:2272] T 1ca231f6e37b4c8f9b87f9b8982da72b P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.701176 17157 tablet_replica.cc:333] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.701782 17157 raft_consensus.cc:2243] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.702785 17157 raft_consensus.cc:2272] T 892f6fa502274e7fb35cc13423c31191 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.705402 17157 tablet_replica.cc:333] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.705947 17157 raft_consensus.cc:2243] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.706445 17157 raft_consensus.cc:2272] T 9cb11ba7ed054b2a8d8c54e65c09683b P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.709477 17157 tablet_replica.cc:333] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.710456 17157 raft_consensus.cc:2243] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.711258 17157 raft_consensus.cc:2272] T c9673f2a3de04ae78ee87b03e8e6e7e2 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.714627 17157 tablet_replica.cc:333] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.715512 17157 raft_consensus.cc:2243] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:21:55.715929 29364 consensus_peers.cc:597] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.716761 17157 raft_consensus.cc:2272] T 6ed2cb7f137e4341b90c5859de219572 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.719784 17157 tablet_replica.cc:333] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.720536 17157 raft_consensus.cc:2243] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.721493 17157 raft_consensus.cc:2272] T f4a56e02dd2f4d6694005ea175134f6a P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.724320 17157 tablet_replica.cc:333] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.725099 17157 raft_consensus.cc:2243] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.726164 17157 raft_consensus.cc:2272] T a4ceaee2df5d459cb758be7b909fed91 P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.728952 17157 tablet_replica.cc:333] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.729502 17157 raft_consensus.cc:2243] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.730073 17157 raft_consensus.cc:2272] T d7de9e42928e47a4908555f387976a6f P 8457574bb2ea439fa9c42cda14d14a6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.732740 17157 tablet_replica.cc:333] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.733343 17157 raft_consensus.cc:2243] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.734036 17157 raft_consensus.cc:2272] T 88521e06c94842ee86e176f041a1762a P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.737648 17157 tablet_replica.cc:333] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.738410 17157 raft_consensus.cc:2243] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.739140 17157 raft_consensus.cc:2272] T f0b49a8b27bd40d49c8460792ae53b55 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.742291 17157 tablet_replica.cc:333] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.742851 17157 raft_consensus.cc:2243] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.743525 17157 raft_consensus.cc:2272] T 600f9682a067439f8959c1272aa64920 P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.746049 17157 tablet_replica.cc:333] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.746563 17157 raft_consensus.cc:2243] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.747602 17157 raft_consensus.cc:2272] T aacfb04bda734b7d90fde9daea58c55e P 8457574bb2ea439fa9c42cda14d14a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.750059 17157 tablet_replica.cc:333] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b: stopping tablet replica
I20260709 12:21:55.750558 17157 raft_consensus.cc:2243] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.751096 17157 raft_consensus.cc:2272] T 6146c6d185674376855062ba856140db P 8457574bb2ea439fa9c42cda14d14a6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.789695 17157 tablet_server.cc:196] TabletServer@127.16.193.66:0 shutdown complete.
W20260709 12:21:55.818938 29438 consensus_peers.cc:597] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.840480 17157 tablet_server.cc:179] TabletServer@127.16.193.67:0 shutting down...
I20260709 12:21:55.883572 17157 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:21:55.884275 29438 consensus_peers.cc:597] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 -> Peer a23cb2cefcbc487a9ae1dc6b201720fd (127.16.193.65:40545): Couldn't send request to peer a23cb2cefcbc487a9ae1dc6b201720fd. Status: Network error: Client connection negotiation failed: client connection to 127.16.193.65:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:55.884635 17157 tablet_replica.cc:333] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.885804 17157 raft_consensus.cc:2243] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.886704 17157 raft_consensus.cc:2272] T 892f6fa502274e7fb35cc13423c31191 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.889501 17157 tablet_replica.cc:333] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.890189 17157 raft_consensus.cc:2243] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.890767 17157 raft_consensus.cc:2272] T 1ca231f6e37b4c8f9b87f9b8982da72b P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.893440 17157 tablet_replica.cc:333] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.894054 17157 raft_consensus.cc:2243] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.894627 17157 raft_consensus.cc:2272] T 9cb4f8e2533c45e8becbb28502629c19 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.897344 17157 tablet_replica.cc:333] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.897965 17157 raft_consensus.cc:2243] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.898599 17157 raft_consensus.cc:2272] T f7775a7515354b7f9ef7fb11f619f128 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.901219 17157 tablet_replica.cc:333] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.901834 17157 raft_consensus.cc:2243] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.902753 17157 raft_consensus.cc:2272] T fe52819efe134429ad0ae95af9369b45 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.905438 17157 tablet_replica.cc:333] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.906071 17157 raft_consensus.cc:2243] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.906574 17157 raft_consensus.cc:2272] T aacfb04bda734b7d90fde9daea58c55e P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.909000 17157 tablet_replica.cc:333] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.909618 17157 raft_consensus.cc:2243] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.910125 17157 raft_consensus.cc:2272] T 67f3135f4bf74024910653e5eddc4c62 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.912838 17157 tablet_replica.cc:333] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.913651 17157 raft_consensus.cc:2243] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.914234 17157 raft_consensus.cc:2272] T fff5eef9749f40118843a09a8958f788 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.916869 17157 tablet_replica.cc:333] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.917455 17157 raft_consensus.cc:2243] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.918418 17157 raft_consensus.cc:2272] T 9cb11ba7ed054b2a8d8c54e65c09683b P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.920909 17157 tablet_replica.cc:333] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.921478 17157 raft_consensus.cc:2243] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.921994 17157 raft_consensus.cc:2272] T c9673f2a3de04ae78ee87b03e8e6e7e2 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.924432 17157 tablet_replica.cc:333] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.925132 17157 raft_consensus.cc:2243] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.926117 17157 raft_consensus.cc:2272] T 88521e06c94842ee86e176f041a1762a P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.928682 17157 tablet_replica.cc:333] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.929359 17157 raft_consensus.cc:2243] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.929903 17157 raft_consensus.cc:2272] T 600f9682a067439f8959c1272aa64920 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.932548 17157 tablet_replica.cc:333] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.933117 17157 raft_consensus.cc:2243] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.933723 17157 raft_consensus.cc:2272] T f0b49a8b27bd40d49c8460792ae53b55 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.936201 17157 tablet_replica.cc:333] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.936745 17157 raft_consensus.cc:2243] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.937361 17157 raft_consensus.cc:2272] T 6146c6d185674376855062ba856140db P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.939987 17157 tablet_replica.cc:333] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.940569 17157 raft_consensus.cc:2243] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.941159 17157 raft_consensus.cc:2272] T 6ed2cb7f137e4341b90c5859de219572 P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.943742 17157 tablet_replica.cc:333] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.944298 17157 raft_consensus.cc:2243] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.944861 17157 raft_consensus.cc:2272] T f4a56e02dd2f4d6694005ea175134f6a P d4dbb6919e7e4472ad819aefad1d45a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.947381 17157 tablet_replica.cc:333] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.947932 17157 raft_consensus.cc:2243] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:55.948493 17157 raft_consensus.cc:2272] T a4ceaee2df5d459cb758be7b909fed91 P d4dbb6919e7e4472ad819aefad1d45a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:55.951139 17157 tablet_replica.cc:333] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0: stopping tablet replica
I20260709 12:21:55.951725 17157 raft_consensus.cc:2243] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:55.952764 17157 raft_consensus.cc:2272] T d7de9e42928e47a4908555f387976a6f P d4dbb6919e7e4472ad819aefad1d45a0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:56.001237 17157 tablet_server.cc:196] TabletServer@127.16.193.67:0 shutdown complete.
I20260709 12:21:56.045588 17157 master.cc:562] Master@127.16.193.126:34249 shutting down...
I20260709 12:21:56.078574 17157 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:56.079497 17157 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:56.079865 17157 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7c5b5e25fdcb42aaa5387c3a0a5676b4: stopping tablet replica
W20260709 12:21:57.097877 17157 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 29253)
I20260709 12:21:58.026149 17157 master.cc:584] Master@127.16.193.126:34249 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (22833 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (43764 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (405575 ms total)
[  PASSED  ] 8 tests.
I20260709 12:21:58.087610 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2239 messages since previous log ~33 seconds ago
I20260709 12:21:58.087877 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2242 messages since previous log ~34 seconds ago
I20260709 12:21:58.088053 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2241 messages since previous log ~34 seconds ago
I20260709 12:21:58.088276 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~52 seconds ago
I20260709 12:21:58.088485 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~52 seconds ago
I20260709 12:21:58.088711 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~136 seconds ago
I20260709 12:21:58.088990 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~2 seconds ago
I20260709 12:21:58.089356 17157 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~21 seconds ago
